[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:03:00 dist-test-slave-tc2s krb5kdc[953](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:03:00 dist-test-slave-tc2s krb5kdc[953](info): set up 2 sockets
Aug 06 15:03:00 dist-test-slave-tc2s krb5kdc[953](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 15:03:02.140842 945 mini_kdc.cc:121] Time spent starting KDC: real 1.323s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:03:02 dist-test-slave-tc2s krb5kdc[953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:03:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:03:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:03:02.267264 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:41705
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:44901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:41705
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:02.364473 976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:02.364677 976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:02.364707 976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:02.365808 976 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:03:02.365846 976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:02.365865 976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:02.365883 976 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:03:02.365917 976 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:03:02.367558 976 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:44901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:41705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:41705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:02.367902 976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:02.368183 976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:02.376098 984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:02.376529 976 server_base.cc:1034] running on GCE node
W20260806 15:03:02.376638 985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:02.376647 987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:02.377010 976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:02.377394 976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:02.378541 976 hybrid_clock.cc:648] HybridClock initialized: now 1786028582378523 us; error 71 us; skew 500 ppm
Aug 06 15:03:02 dist-test-slave-tc2s krb5kdc[953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028582, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:02.385303 976 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:03:02.385869 976 webserver.cc:460] Webserver started at http://127.0.236.126:38357/ using document root <none> and password file <none>
I20260806 15:03:02.386161 976 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:02.386214 976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:02.386350 976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:02.387653 976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "0247366906fa47649f48a4c49a5b4d54"
format_stamp: "Formatted at 2026-08-06 15:03:02 on dist-test-slave-tc2s"
server_key: "64eae65c03a5f23b8cdca740932684f0"
server_key_iv: "3ce1b5db831a2b49b2597a455c843d44"
server_key_version: "encryptionkey@0"
I20260806 15:03:02.388399 976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "0247366906fa47649f48a4c49a5b4d54"
format_stamp: "Formatted at 2026-08-06 15:03:02 on dist-test-slave-tc2s"
server_key: "64eae65c03a5f23b8cdca740932684f0"
server_key_iv: "3ce1b5db831a2b49b2597a455c843d44"
server_key_version: "encryptionkey@0"
I20260806 15:03:02.390158 976 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 15:03:02.391155 996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:02.391386 976 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:02.391523 976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "0247366906fa47649f48a4c49a5b4d54"
format_stamp: "Formatted at 2026-08-06 15:03:02 on dist-test-slave-tc2s"
server_key: "64eae65c03a5f23b8cdca740932684f0"
server_key_iv: "3ce1b5db831a2b49b2597a455c843d44"
server_key_version: "encryptionkey@0"
I20260806 15:03:02.391592 976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:02.406591 976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:02.408628 976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:02.408802 976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:02.414072 976 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:41705
I20260806 15:03:02.414135 1074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:41705 every 8 connection(s)
I20260806 15:03:02.414709 976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:03:02.416008 1075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:02.419173 1075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0247366906fa47649f48a4c49a5b4d54: Bootstrap starting.
I20260806 15:03:02.420120 1075 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0247366906fa47649f48a4c49a5b4d54: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:02.420513 1075 log.cc:824] T 00000000000000000000000000000000 P 0247366906fa47649f48a4c49a5b4d54: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:02.421327 1075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0247366906fa47649f48a4c49a5b4d54: No bootstrap required, opened a new log
I20260806 15:03:02.421540 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 976
I20260806 15:03:02.421629 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:03:02.422302 945 external_mini_cluster.cc:1428] Setting key 4ec0cc76298fd811a6f68d6ab90caeda
I20260806 15:03:02.423480 1075 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0247366906fa47649f48a4c49a5b4d54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0247366906fa47649f48a4c49a5b4d54" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41705 } }
I20260806 15:03:02.423753 1075 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0247366906fa47649f48a4c49a5b4d54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:02.423868 1075 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0247366906fa47649f48a4c49a5b4d54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0247366906fa47649f48a4c49a5b4d54, State: Initialized, Role: FOLLOWER
I20260806 15:03:02.424033 1075 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0247366906fa47649f48a4c49a5b4d54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0247366906fa47649f48a4c49a5b4d54" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41705 } }
I20260806 15:03:02.424187 1075 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0247366906fa47649f48a4c49a5b4d54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:03:02.424226 1075 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0247366906fa47649f48a4c49a5b4d54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:03:02.424278 1075 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0247366906fa47649f48a4c49a5b4d54 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:02.425158 1075 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0247366906fa47649f48a4c49a5b4d54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0247366906fa47649f48a4c49a5b4d54" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41705 } }
I20260806 15:03:02.425346 1075 leader_election.cc:302] T 00000000000000000000000000000000 P 0247366906fa47649f48a4c49a5b4d54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0247366906fa47649f48a4c49a5b4d54; no voters:
I20260806 15:03:02.425570 1075 leader_election.cc:288] T 00000000000000000000000000000000 P 0247366906fa47649f48a4c49a5b4d54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:03:02.425606 1081 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0247366906fa47649f48a4c49a5b4d54 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:02.425735 1081 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0247366906fa47649f48a4c49a5b4d54 [term 1 LEADER]: Becoming Leader. State: Replica: 0247366906fa47649f48a4c49a5b4d54, State: Running, Role: LEADER
I20260806 15:03:02.425879 1081 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0247366906fa47649f48a4c49a5b4d54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0247366906fa47649f48a4c49a5b4d54" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41705 } }
I20260806 15:03:02.426095 1075 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0247366906fa47649f48a4c49a5b4d54 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:03:02.426401 1082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0247366906fa47649f48a4c49a5b4d54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0247366906fa47649f48a4c49a5b4d54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0247366906fa47649f48a4c49a5b4d54" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41705 } } }
I20260806 15:03:02.426478 1082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0247366906fa47649f48a4c49a5b4d54 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:02.426616 1083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0247366906fa47649f48a4c49a5b4d54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0247366906fa47649f48a4c49a5b4d54. Latest consensus state: current_term: 1 leader_uuid: "0247366906fa47649f48a4c49a5b4d54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0247366906fa47649f48a4c49a5b4d54" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41705 } } }
I20260806 15:03:02.426677 1083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0247366906fa47649f48a4c49a5b4d54 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:02.427268 1085 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:03:02.428637 1085 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:03:02.430531 1085 catalog_manager.cc:1347] Generated new cluster ID: 677546ed24ca41a1bbb7dce3c4694aa0
I20260806 15:03:02.430580 1085 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 15:03:02 dist-test-slave-tc2s krb5kdc[953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:02.437513 1085 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:03:02.438079 1085 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:03:02.440898 1085 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0247366906fa47649f48a4c49a5b4d54: Generated new TSK 0
I20260806 15:03:02.441100 1085 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:03:02.444023 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:02.423569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57974 (local address 127.0.236.126:41705)
0806 15:03:02.423931 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:02.423955 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:02.424122 (+ 167us) server_negotiation.cc:407] Connection header received
0806 15:03:02.426065 (+ 1943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:02.426072 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:02.426231 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:02.426366 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:02.428651 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:02.430553 (+ 1902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:02.432206 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:02.432898 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:02.433050 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:02.437206 (+ 4156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:02.437210 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:02.437210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:02.437221 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:02.442183 (+ 4962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:02.442587 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:02.442589 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:02.442590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:02.442706 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:02.442927 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:02.442928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:02.442929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:02.443199 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:02.443330 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:02.443552 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 15:03:02.443751 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":114,"threads_started":1}
WARNING: no policy specified for kudu/127.0.236.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
I20260806 15:03:02.470152 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:0
--local_ip_for_outbound_sockets=127.0.236.65
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:41705
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:44901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:02.552784 1113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:02.552984 1113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:02.553007 1113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:02.554189 1113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:02.554234 1113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:02.554260 1113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:03:02.555850 1113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:44901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:41705
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:02.556159 1113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:02.556391 1113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:02.562449 1119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:02.562453 1118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:02.562443 1121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:02.562791 1113 server_base.cc:1034] running on GCE node
I20260806 15:03:02.563012 1113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:02.563294 1113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:02.564471 1113 hybrid_clock.cc:648] HybridClock initialized: now 1786028582564452 us; error 32 us; skew 500 ppm
Aug 06 15:03:02 dist-test-slave-tc2s krb5kdc[953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028582, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:02.567523 1113 init.cc:377] Logged in from keytab as kudu/127.0.236.65@KRBTEST.COM (short username kudu)
I20260806 15:03:02.568069 1113 webserver.cc:460] Webserver started at http://127.0.236.65:39369/ using document root <none> and password file <none>
I20260806 15:03:02.568315 1113 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:02.568370 1113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:02.568491 1113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:02.569976 1113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-0/data/instance:
uuid: "4bda844a315c4a50aaf2d95829f5b494"
format_stamp: "Formatted at 2026-08-06 15:03:02 on dist-test-slave-tc2s"
server_key: "fd0e5b3ddae5f4f6376f01b8559dc72a"
server_key_iv: "1f65229e57c3b8ffbd094ef06402772d"
server_key_version: "encryptionkey@0"
I20260806 15:03:02.570410 1113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-0/wal/instance:
uuid: "4bda844a315c4a50aaf2d95829f5b494"
format_stamp: "Formatted at 2026-08-06 15:03:02 on dist-test-slave-tc2s"
server_key: "fd0e5b3ddae5f4f6376f01b8559dc72a"
server_key_iv: "1f65229e57c3b8ffbd094ef06402772d"
server_key_version: "encryptionkey@0"
I20260806 15:03:02.572125 1113 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 15:03:02.573104 1132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:02.573292 1113 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:03:02.573376 1113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "4bda844a315c4a50aaf2d95829f5b494"
format_stamp: "Formatted at 2026-08-06 15:03:02 on dist-test-slave-tc2s"
server_key: "fd0e5b3ddae5f4f6376f01b8559dc72a"
server_key_iv: "1f65229e57c3b8ffbd094ef06402772d"
server_key_version: "encryptionkey@0"
I20260806 15:03:02.573429 1113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:02.595176 1113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:02.597153 1113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:02.597281 1113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:02.597527 1113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:02.597849 1113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:02.597980 1113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:02.598021 1113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:02.598045 1113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:02.608675 1113 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:46167
I20260806 15:03:02.608714 1298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:46167 every 8 connection(s)
I20260806 15:03:02.608999 1113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
Aug 06 15:03:02 dist-test-slave-tc2s krb5kdc[953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028582, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:02.615513 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 1113
I20260806 15:03:02.615602 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-0/wal/instance
I20260806 15:03:02.615742 945 external_mini_cluster.cc:1428] Setting key d7247117f0cfdedc1d452b927fb7ed00
I20260806 15:03:02.617923 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:02.609882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:58829 (local address 127.0.236.126:41705)
0806 15:03:02.609949 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:02.609951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:02.610135 (+ 184us) server_negotiation.cc:407] Connection header received
0806 15:03:02.610582 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:02.610583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:02.610592 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:02.610644 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:02.611637 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:02.612127 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:02.613017 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:02.613198 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:02.613241 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:02.615099 (+ 1858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:02.615101 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:02.615101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:02.615108 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:02.616554 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:02.616982 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:02.616982 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:02.616982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:02.617031 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:02.617267 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:02.617267 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:02.617267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:02.617376 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:02.617422 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:02.617746 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 15:03:02.617821 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 15:03:02.618116 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:02.609835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:41705 (local address 127.0.236.65:58829)
0806 15:03:02.610016 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:02.610058 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:02.610507 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:02.610694 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:02.610697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:02.611380 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:02.611567 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:02.611572 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:02.612188 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:02.612188 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:02.612958 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:02.612960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:02.613249 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:02.613250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:02.613345 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:02.613722 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:02.613729 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:02.615011 (+ 1282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:02.616627 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:02.616634 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:02.616636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:02.616919 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:02.617079 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:02.617079 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:02.617080 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:02.617209 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:02.617430 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:02.617437 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:02.617610 (+ 173us) client_negotiation.cc:769] Sending connection context
0806 15:03:02.617730 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 15:03:02.617886 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":111,"threads_started":1}
I20260806 15:03:02.618657 1299 heartbeater.cc:347] Connected to a master server at 127.0.236.126:41705
I20260806 15:03:02.618741 1299 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:02.618916 1299 heartbeater.cc:511] Master 127.0.236.126:41705 requested a full tablet report, sending...
I20260806 15:03:02.619705 1024 ts_manager.cc:194] Registered new tserver with Master: 4bda844a315c4a50aaf2d95829f5b494 (127.0.236.65:46167)
I20260806 15:03:02.620393 1024 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.65@KRBTEST.COM'} at 127.0.236.65:58829
WARNING: no policy specified for kudu/127.0.236.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
I20260806 15:03:02.642107 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:0
--local_ip_for_outbound_sockets=127.0.236.66
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:41705
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:44901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:02.722594 1308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:02.722780 1308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:02.722801 1308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:02.723908 1308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:02.723949 1308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:02.723977 1308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:03:02.725380 1308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:44901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:41705
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:02.725602 1308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:02.725797 1308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:02.730492 1317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:02.730481 1315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:02.730484 1314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:02.730707 1308 server_base.cc:1034] running on GCE node
I20260806 15:03:02.730928 1308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:02.731148 1308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:02.732282 1308 hybrid_clock.cc:648] HybridClock initialized: now 1786028582732269 us; error 31 us; skew 500 ppm
Aug 06 15:03:02 dist-test-slave-tc2s krb5kdc[953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028582, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:02.734627 1308 init.cc:377] Logged in from keytab as kudu/127.0.236.66@KRBTEST.COM (short username kudu)
I20260806 15:03:02.735078 1308 webserver.cc:460] Webserver started at http://127.0.236.66:45659/ using document root <none> and password file <none>
I20260806 15:03:02.735273 1308 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:02.735319 1308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:02.735443 1308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:02.736395 1308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-1/data/instance:
uuid: "fe2e3e1559004d739fd07e35e9b2a955"
format_stamp: "Formatted at 2026-08-06 15:03:02 on dist-test-slave-tc2s"
server_key: "9f5ad220498eebd2b8fd0f02532255f2"
server_key_iv: "ba8309301157626f30258aa975a3f383"
server_key_version: "encryptionkey@0"
I20260806 15:03:02.736714 1308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-1/wal/instance:
uuid: "fe2e3e1559004d739fd07e35e9b2a955"
format_stamp: "Formatted at 2026-08-06 15:03:02 on dist-test-slave-tc2s"
server_key: "9f5ad220498eebd2b8fd0f02532255f2"
server_key_iv: "ba8309301157626f30258aa975a3f383"
server_key_version: "encryptionkey@0"
I20260806 15:03:02.737882 1308 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:03:02.738591 1324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:02.738737 1308 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:03:02.738818 1308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "fe2e3e1559004d739fd07e35e9b2a955"
format_stamp: "Formatted at 2026-08-06 15:03:02 on dist-test-slave-tc2s"
server_key: "9f5ad220498eebd2b8fd0f02532255f2"
server_key_iv: "ba8309301157626f30258aa975a3f383"
server_key_version: "encryptionkey@0"
I20260806 15:03:02.738884 1308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:02.757882 1308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:02.759511 1308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:02.759637 1308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:02.759830 1308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:02.760099 1308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:02.760133 1308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:02.760159 1308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:02.760176 1308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:02.769507 1308 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:45231
I20260806 15:03:02.769542 1488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:45231 every 8 connection(s)
I20260806 15:03:02.769822 1308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
Aug 06 15:03:02 dist-test-slave-tc2s krb5kdc[953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028582, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:02.777204 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 1308
I20260806 15:03:02.777302 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-1/wal/instance
I20260806 15:03:02.777498 945 external_mini_cluster.cc:1428] Setting key b570f80a63a4c1f892d7252879087fd8
I20260806 15:03:02.777868 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:02.770528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:43211 (local address 127.0.236.126:41705)
0806 15:03:02.770572 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:02.770574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:02.770773 (+ 199us) server_negotiation.cc:407] Connection header received
0806 15:03:02.771202 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:02.771203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:02.771212 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:02.771261 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:02.772095 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:02.772571 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:02.773461 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:02.773632 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:02.773674 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:02.775474 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:02.775475 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:02.775475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:02.775482 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:02.776564 (+ 1082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:02.776894 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:02.776895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:02.776895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:02.776946 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:02.777176 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:02.777177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:02.777177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:02.777315 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:02.777379 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:02.777723 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 15:03:02.777801 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 15:03:02.778074 1492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:02.770499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:41705 (local address 127.0.236.66:43211)
0806 15:03:02.770655 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:02.770693 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:02.771127 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:02.771311 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:02.771313 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:02.771841 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:02.772028 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:02.772033 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:02.772630 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:02.772630 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:02.773402 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:02.773404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:02.773682 (+ 278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:02.773682 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:02.773770 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:02.774172 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:02.774179 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:02.775388 (+ 1209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:02.776630 (+ 1242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:02.776637 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:02.776638 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:02.776839 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:02.776981 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:02.776981 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:02.776981 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:02.777117 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:02.777419 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:02.777426 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:02.777577 (+ 151us) client_negotiation.cc:769] Sending connection context
0806 15:03:02.777690 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 15:03:02.777841 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
I20260806 15:03:02.778586 1489 heartbeater.cc:347] Connected to a master server at 127.0.236.126:41705
I20260806 15:03:02.778680 1489 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:02.778855 1489 heartbeater.cc:511] Master 127.0.236.126:41705 requested a full tablet report, sending...
I20260806 15:03:02.779261 1024 ts_manager.cc:194] Registered new tserver with Master: fe2e3e1559004d739fd07e35e9b2a955 (127.0.236.66:45231)
I20260806 15:03:02.779615 1024 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.66@KRBTEST.COM'} at 127.0.236.66:43211
WARNING: no policy specified for kudu/127.0.236.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
I20260806 15:03:02.802433 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:0
--local_ip_for_outbound_sockets=127.0.236.67
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:41705
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:44901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:02.885010 1497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:02.885183 1497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:02.885205 1497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:02.886292 1497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:02.886332 1497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:02.886358 1497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:03:02.887781 1497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:44901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:41705
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:02.887986 1497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:02.888177 1497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:02.892935 1503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:02.892910 1505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:02.892948 1497 server_base.cc:1034] running on GCE node
W20260806 15:03:02.892910 1507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:02.893308 1497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:02.893527 1497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:02.894656 1497 hybrid_clock.cc:648] HybridClock initialized: now 1786028582894644 us; error 30 us; skew 500 ppm
Aug 06 15:03:02 dist-test-slave-tc2s krb5kdc[953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028582, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:02.896946 1497 init.cc:377] Logged in from keytab as kudu/127.0.236.67@KRBTEST.COM (short username kudu)
I20260806 15:03:02.897415 1497 webserver.cc:460] Webserver started at http://127.0.236.67:33917/ using document root <none> and password file <none>
I20260806 15:03:02.897629 1497 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:02.897675 1497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:02.897794 1497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:02.898754 1497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-2/data/instance:
uuid: "7820923b0d354b10a919066f46b0bbc4"
format_stamp: "Formatted at 2026-08-06 15:03:02 on dist-test-slave-tc2s"
server_key: "0eb42a48a5ba272ffc3fa3bde493f22c"
server_key_iv: "b51b039ba9728a24948f516d3b7d90d2"
server_key_version: "encryptionkey@0"
I20260806 15:03:02.899066 1497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-2/wal/instance:
uuid: "7820923b0d354b10a919066f46b0bbc4"
format_stamp: "Formatted at 2026-08-06 15:03:02 on dist-test-slave-tc2s"
server_key: "0eb42a48a5ba272ffc3fa3bde493f22c"
server_key_iv: "b51b039ba9728a24948f516d3b7d90d2"
server_key_version: "encryptionkey@0"
I20260806 15:03:02.900202 1497 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:03:02.900818 1519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:02.900951 1497 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:02.901028 1497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "7820923b0d354b10a919066f46b0bbc4"
format_stamp: "Formatted at 2026-08-06 15:03:02 on dist-test-slave-tc2s"
server_key: "0eb42a48a5ba272ffc3fa3bde493f22c"
server_key_iv: "b51b039ba9728a24948f516d3b7d90d2"
server_key_version: "encryptionkey@0"
I20260806 15:03:02.901087 1497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:02.913380 1497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:02.915169 1497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:02.915293 1497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:02.915482 1497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:02.915732 1497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:02.915764 1497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:02.915789 1497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:02.915814 1497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:02.925109 1497 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:37345
I20260806 15:03:02.925146 1688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:37345 every 8 connection(s)
I20260806 15:03:02.925431 1497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
I20260806 15:03:02.927892 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 1497
I20260806 15:03:02.927980 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028580791223-945-0/minicluster-data/ts-2/wal/instance
I20260806 15:03:02.928135 945 external_mini_cluster.cc:1428] Setting key 249e00628f900d05d6158997ceb9d806
Aug 06 15:03:02 dist-test-slave-tc2s krb5kdc[953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028582, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:02.933980 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:02.926165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:34071 (local address 127.0.236.126:41705)
0806 15:03:02.926209 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:02.926211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:02.926394 (+ 183us) server_negotiation.cc:407] Connection header received
0806 15:03:02.926998 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:02.927000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:02.927012 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:02.927054 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:02.927936 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:02.928531 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:02.929709 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:02.929940 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:02.929998 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:02.931910 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:02.931911 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:02.931911 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:02.931917 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:02.932959 (+ 1042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:02.933281 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:02.933282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:02.933282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:02.933326 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:02.933496 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:02.933496 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:02.933497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:02.933589 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:02.933631 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:02.933820 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 15:03:02.933883 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:03:02.934157 1694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:02.926147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:41705 (local address 127.0.236.67:34071)
0806 15:03:02.926285 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:02.926315 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:02.926891 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:02.927110 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:02.927113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:02.927626 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:02.927848 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:02.927852 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:02.928619 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:02.928620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:02.929645 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:02.929648 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:02.930001 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:02.930002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:02.930104 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:02.930555 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:02.930562 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:02.931818 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:02.933021 (+ 1203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:02.933026 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:02.933028 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:02.933228 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:02.933368 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:02.933368 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:02.933369 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:02.933455 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:02.933629 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:02.933632 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:02.933732 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 15:03:02.933803 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 15:03:02.933956 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":82,"threads_started":1}
I20260806 15:03:02.934544 1692 heartbeater.cc:347] Connected to a master server at 127.0.236.126:41705
I20260806 15:03:02.934618 1692 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:02.934772 1692 heartbeater.cc:511] Master 127.0.236.126:41705 requested a full tablet report, sending...
I20260806 15:03:02.935091 1024 ts_manager.cc:194] Registered new tserver with Master: 7820923b0d354b10a919066f46b0bbc4 (127.0.236.67:37345)
I20260806 15:03:02.935454 1024 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.67@KRBTEST.COM'} at 127.0.236.67:34071
I20260806 15:03:02.940011 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 15:03:02 dist-test-slave-tc2s krb5kdc[953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028582, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 15:03:02 dist-test-slave-tc2s krb5kdc[953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028582, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.236.65@KRBTEST.COM
I20260806 15:03:02.964090 1700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:02.957297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60804 (local address 127.0.236.65:46167)
0806 15:03:02.957518 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:02.957522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:02.957533 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:03:02.957556 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:02.957558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:02.957571 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:02.957637 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:02.958170 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:02.958926 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:02.959584 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:02.959882 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:02.959930 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:02.961807 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:02.961815 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:02.961815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:02.961822 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:02.963234 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:02.963493 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:02.963495 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:02.963495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:02.963547 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:02.963666 (+ 119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:02.963666 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:02.963667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:02.963775 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:02.963863 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:02.963922 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 15:03:02.963983 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":137,"threads_started":1}
W20260806 15:03:02.964540 1173 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60804
Aug 06 15:03:02 dist-test-slave-tc2s krb5kdc[953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 15:03:02 dist-test-slave-tc2s krb5kdc[953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.65@KRBTEST.COM
I20260806 15:03:02.985402 1700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:02.979516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60810 (local address 127.0.236.65:46167)
0806 15:03:02.979588 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:02.979590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:02.979662 (+ 72us) server_negotiation.cc:407] Connection header received
0806 15:03:02.979720 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:02.979721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:02.979730 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:02.979777 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:02.980271 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:02.980738 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:02.981399 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:02.981592 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:02.981644 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:02.983498 (+ 1854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:02.983500 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:02.983500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:02.983507 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:02.984493 (+ 986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:02.984773 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:02.984774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:02.984774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:02.984822 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:02.984980 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:02.984981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:02.984981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:02.985069 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:02.985161 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:02.985247 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 15:03:02.985319 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 15:03:02.985836 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 1113
I20260806 15:03:02.995114 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 1308
I20260806 15:03:03.013959 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 1497
I20260806 15:03:03.020226 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 976
2026-08-06T15:03:03Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2233 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:03:03 dist-test-slave-tc2s krb5kdc[1710](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:03:03 dist-test-slave-tc2s krb5kdc[1710](info): set up 2 sockets
Aug 06 15:03:03 dist-test-slave-tc2s krb5kdc[1710](info): commencing operation
krb5kdc: starting...
W20260806 15:03:04.124369 945 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:03:04 dist-test-slave-tc2s krb5kdc[1710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:03:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:03:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:03:04.232576 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:46513
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:37377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:46513
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:04.310503 1730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:04.310698 1730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:04.310729 1730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:04.311822 1730 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:03:04.311861 1730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:04.311880 1730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:04.311895 1730 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:03:04.311911 1730 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:03:04.313357 1730 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:37377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:46513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:46513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:04.313587 1730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:04.313781 1730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:04.318248 1737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:04.318331 1730 server_base.cc:1034] running on GCE node
W20260806 15:03:04.318267 1739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:04.318267 1735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:04.318660 1730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:04.318909 1730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:04.320034 1730 hybrid_clock.cc:648] HybridClock initialized: now 1786028584320026 us; error 30 us; skew 500 ppm
Aug 06 15:03:04 dist-test-slave-tc2s krb5kdc[1710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028584, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:04.322337 1730 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:03:04.322764 1730 webserver.cc:460] Webserver started at http://127.0.236.126:36161/ using document root <none> and password file <none>
I20260806 15:03:04.322952 1730 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:04.322999 1730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:04.323122 1730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:04.324083 1730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "ce9db776e70646f980b34983c0f39c40"
format_stamp: "Formatted at 2026-08-06 15:03:04 on dist-test-slave-tc2s"
server_key: "30e54a5bf1424ddccc68d553be86233c"
server_key_iv: "99c50804653d86b541d4d29e385f36f8"
server_key_version: "encryptionkey@0"
I20260806 15:03:04.324394 1730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "ce9db776e70646f980b34983c0f39c40"
format_stamp: "Formatted at 2026-08-06 15:03:04 on dist-test-slave-tc2s"
server_key: "30e54a5bf1424ddccc68d553be86233c"
server_key_iv: "99c50804653d86b541d4d29e385f36f8"
server_key_version: "encryptionkey@0"
I20260806 15:03:04.325556 1730 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:03:04.326193 1749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:04.326337 1730 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:04.326419 1730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "ce9db776e70646f980b34983c0f39c40"
format_stamp: "Formatted at 2026-08-06 15:03:04 on dist-test-slave-tc2s"
server_key: "30e54a5bf1424ddccc68d553be86233c"
server_key_iv: "99c50804653d86b541d4d29e385f36f8"
server_key_version: "encryptionkey@0"
I20260806 15:03:04.326483 1730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:04.365232 1730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:04.366927 1730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:04.367056 1730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:04.370954 1730 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:46513
I20260806 15:03:04.370999 1821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:46513 every 8 connection(s)
I20260806 15:03:04.371307 1730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:03:04.372102 1822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:04.374365 1822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce9db776e70646f980b34983c0f39c40: Bootstrap starting.
I20260806 15:03:04.374908 1822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce9db776e70646f980b34983c0f39c40: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:04.375164 1822 log.cc:824] T 00000000000000000000000000000000 P ce9db776e70646f980b34983c0f39c40: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:04.375770 1822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce9db776e70646f980b34983c0f39c40: No bootstrap required, opened a new log
I20260806 15:03:04.376595 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 1730
I20260806 15:03:04.376685 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:03:04.376837 945 external_mini_cluster.cc:1428] Setting key 1acf6071db6867f6e642ff7994ac0916
I20260806 15:03:04.377668 1822 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce9db776e70646f980b34983c0f39c40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9db776e70646f980b34983c0f39c40" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46513 } }
I20260806 15:03:04.377823 1822 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce9db776e70646f980b34983c0f39c40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:04.377861 1822 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce9db776e70646f980b34983c0f39c40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce9db776e70646f980b34983c0f39c40, State: Initialized, Role: FOLLOWER
I20260806 15:03:04.378012 1822 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce9db776e70646f980b34983c0f39c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9db776e70646f980b34983c0f39c40" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46513 } }
I20260806 15:03:04.378096 1822 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce9db776e70646f980b34983c0f39c40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:03:04.378134 1822 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce9db776e70646f980b34983c0f39c40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:03:04.378182 1822 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce9db776e70646f980b34983c0f39c40 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:04.379122 1822 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce9db776e70646f980b34983c0f39c40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9db776e70646f980b34983c0f39c40" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46513 } }
I20260806 15:03:04.379210 1822 leader_election.cc:302] T 00000000000000000000000000000000 P ce9db776e70646f980b34983c0f39c40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce9db776e70646f980b34983c0f39c40; no voters:
I20260806 15:03:04.379350 1822 leader_election.cc:288] T 00000000000000000000000000000000 P ce9db776e70646f980b34983c0f39c40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:03:04.379415 1832 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce9db776e70646f980b34983c0f39c40 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:04.379591 1822 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce9db776e70646f980b34983c0f39c40 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:03:04.380370 1832 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce9db776e70646f980b34983c0f39c40 [term 1 LEADER]: Becoming Leader. State: Replica: ce9db776e70646f980b34983c0f39c40, State: Running, Role: LEADER
I20260806 15:03:04.380481 1832 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce9db776e70646f980b34983c0f39c40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9db776e70646f980b34983c0f39c40" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46513 } }
I20260806 15:03:04.380862 1836 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9db776e70646f980b34983c0f39c40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce9db776e70646f980b34983c0f39c40. Latest consensus state: current_term: 1 leader_uuid: "ce9db776e70646f980b34983c0f39c40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9db776e70646f980b34983c0f39c40" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46513 } } }
I20260806 15:03:04.380968 1836 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9db776e70646f980b34983c0f39c40 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:04.380841 1833 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9db776e70646f980b34983c0f39c40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce9db776e70646f980b34983c0f39c40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9db776e70646f980b34983c0f39c40" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46513 } } }
I20260806 15:03:04.381170 1833 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9db776e70646f980b34983c0f39c40 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:04.381291 1845 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:03:04.381760 1845 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 15:03:04 dist-test-slave-tc2s krb5kdc[1710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:04.383544 1845 catalog_manager.cc:1347] Generated new cluster ID: f3cec3e78cec436e925749103bb03e05
I20260806 15:03:04.383599 1845 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:03:04.387291 1830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:04.377322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45846 (local address 127.0.236.126:46513)
0806 15:03:04.377473 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:04.377481 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:04.377506 (+ 25us) server_negotiation.cc:407] Connection header received
0806 15:03:04.377628 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:04.377633 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:04.377673 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:04.377773 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:04.378518 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.379486 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.380233 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.380898 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.381296 (+ 398us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:04.383809 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:04.383813 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:04.383813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:04.383824 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:04.385687 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:04.386088 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:04.386091 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:04.386091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:04.386211 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:04.386390 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:04.386391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:04.386391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:04.386694 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:04.386868 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:04.386948 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 15:03:04.387081 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":61,"threads_started":1}
I20260806 15:03:04.418419 1845 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:03:04.419191 1845 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:03:04.427407 1845 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce9db776e70646f980b34983c0f39c40: Generated new TSK 0
I20260806 15:03:04.427572 1845 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.236.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
I20260806 15:03:04.453089 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:0
--local_ip_for_outbound_sockets=127.0.236.65
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:46513
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:37377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:04.531957 1861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:04.532145 1861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:04.532178 1861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:04.533279 1861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:04.533318 1861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:04.533353 1861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:03:04.534765 1861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:37377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:46513
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:04.534981 1861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:04.535173 1861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 15:03:04.539805 1861 server_base.cc:1034] running on GCE node
W20260806 15:03:04.539786 1869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:04.539786 1867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:04.539783 1871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:04.540181 1861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:04.540413 1861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:04.541553 1861 hybrid_clock.cc:648] HybridClock initialized: now 1786028584541540 us; error 29 us; skew 500 ppm
Aug 06 15:03:04 dist-test-slave-tc2s krb5kdc[1710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028584, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:04.543905 1861 init.cc:377] Logged in from keytab as kudu/127.0.236.65@KRBTEST.COM (short username kudu)
I20260806 15:03:04.544417 1861 webserver.cc:460] Webserver started at http://127.0.236.65:39823/ using document root <none> and password file <none>
I20260806 15:03:04.544610 1861 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:04.544652 1861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:04.544772 1861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:04.545714 1861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-0/data/instance:
uuid: "b49685f1bc644ef4abf906abd2a28670"
format_stamp: "Formatted at 2026-08-06 15:03:04 on dist-test-slave-tc2s"
server_key: "19c8edc4d66926acee5ec2bc0327062e"
server_key_iv: "bfec511de7ea04315b9944fcd47e68b1"
server_key_version: "encryptionkey@0"
I20260806 15:03:04.546053 1861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-0/wal/instance:
uuid: "b49685f1bc644ef4abf906abd2a28670"
format_stamp: "Formatted at 2026-08-06 15:03:04 on dist-test-slave-tc2s"
server_key: "19c8edc4d66926acee5ec2bc0327062e"
server_key_iv: "bfec511de7ea04315b9944fcd47e68b1"
server_key_version: "encryptionkey@0"
I20260806 15:03:04.547292 1861 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:03:04.547905 1881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:04.548066 1861 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:03:04.548141 1861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "b49685f1bc644ef4abf906abd2a28670"
format_stamp: "Formatted at 2026-08-06 15:03:04 on dist-test-slave-tc2s"
server_key: "19c8edc4d66926acee5ec2bc0327062e"
server_key_iv: "bfec511de7ea04315b9944fcd47e68b1"
server_key_version: "encryptionkey@0"
I20260806 15:03:04.548203 1861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:04.554874 1861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:04.556385 1861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:04.556501 1861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:04.556684 1861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:04.556932 1861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:04.556963 1861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:04.556991 1861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:04.557014 1861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:04.562395 1861 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:34579
I20260806 15:03:04.562438 2048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:34579 every 8 connection(s)
I20260806 15:03:04.562691 1861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:03:04.567463 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 1861
I20260806 15:03:04.567549 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-0/wal/instance
I20260806 15:03:04.567685 945 external_mini_cluster.cc:1428] Setting key 33e2c7eefc430c86c474e896290d2c04
Aug 06 15:03:04 dist-test-slave-tc2s krb5kdc[1710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028584, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:04.571091 1830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:04.563315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:41811 (local address 127.0.236.126:46513)
0806 15:03:04.563373 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:04.563375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:04.563529 (+ 154us) server_negotiation.cc:407] Connection header received
0806 15:03:04.563957 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:04.563958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:04.563973 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:04.564024 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:04.564888 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.565384 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.566334 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.566556 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.566611 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:04.568659 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:04.568660 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:04.568660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:04.568667 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:04.569878 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:04.570247 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:04.570248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:04.570249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:04.570301 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:04.570511 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:04.570512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:04.570512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:04.570618 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:04.570671 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:04.570915 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 15:03:04.571002 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 15:03:04.571221 2052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:04.563275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:46513 (local address 127.0.236.65:41811)
0806 15:03:04.563429 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:04.563460 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:04.563869 (+ 409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:04.564080 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:04.564083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:04.564617 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:04.564789 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:04.564793 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.565463 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.565465 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:04.566256 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:04.566259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.566623 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.566625 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:04.566737 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:04.567147 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:04.567155 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:04.568571 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:04.569970 (+ 1399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:04.569978 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:04.569980 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:04.570182 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:04.570352 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:04.570353 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:04.570353 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:04.570456 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:04.570670 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:04.570676 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:04.570800 (+ 124us) client_negotiation.cc:769] Sending connection context
0806 15:03:04.570889 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 15:03:04.571028 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
I20260806 15:03:04.571636 2049 heartbeater.cc:347] Connected to a master server at 127.0.236.126:46513
I20260806 15:03:04.571715 2049 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:04.571866 2049 heartbeater.cc:511] Master 127.0.236.126:46513 requested a full tablet report, sending...
I20260806 15:03:04.572319 1771 ts_manager.cc:194] Registered new tserver with Master: b49685f1bc644ef4abf906abd2a28670 (127.0.236.65:34579)
I20260806 15:03:04.572927 1771 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.65@KRBTEST.COM'} at 127.0.236.65:41811
WARNING: no policy specified for kudu/127.0.236.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
I20260806 15:03:04.592589 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:0
--local_ip_for_outbound_sockets=127.0.236.66
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:46513
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:37377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:04.671833 2055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:04.672019 2055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:04.672040 2055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:04.673168 2055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:04.673210 2055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:04.673236 2055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:03:04.674654 2055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:37377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:46513
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:04.674861 2055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:04.675057 2055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:04.679584 2062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:04.679723 2055 server_base.cc:1034] running on GCE node
W20260806 15:03:04.679595 2066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:04.679563 2064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:04.680068 2055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:04.680306 2055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:04.681461 2055 hybrid_clock.cc:648] HybridClock initialized: now 1786028584681453 us; error 27 us; skew 500 ppm
Aug 06 15:03:04 dist-test-slave-tc2s krb5kdc[1710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028584, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:04.683756 2055 init.cc:377] Logged in from keytab as kudu/127.0.236.66@KRBTEST.COM (short username kudu)
I20260806 15:03:04.684181 2055 webserver.cc:460] Webserver started at http://127.0.236.66:34653/ using document root <none> and password file <none>
I20260806 15:03:04.684370 2055 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:04.684415 2055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:04.684535 2055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:04.685525 2055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-1/data/instance:
uuid: "6d41695a35a949b8bad075cd8b883be1"
format_stamp: "Formatted at 2026-08-06 15:03:04 on dist-test-slave-tc2s"
server_key: "605880babb9aed96b9dcb43c66c92315"
server_key_iv: "ec5b2a0cda8520b5a1026b8ad30f4e86"
server_key_version: "encryptionkey@0"
I20260806 15:03:04.685838 2055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-1/wal/instance:
uuid: "6d41695a35a949b8bad075cd8b883be1"
format_stamp: "Formatted at 2026-08-06 15:03:04 on dist-test-slave-tc2s"
server_key: "605880babb9aed96b9dcb43c66c92315"
server_key_iv: "ec5b2a0cda8520b5a1026b8ad30f4e86"
server_key_version: "encryptionkey@0"
I20260806 15:03:04.687021 2055 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:03:04.687741 2077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:04.687904 2055 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:03:04.687994 2055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "6d41695a35a949b8bad075cd8b883be1"
format_stamp: "Formatted at 2026-08-06 15:03:04 on dist-test-slave-tc2s"
server_key: "605880babb9aed96b9dcb43c66c92315"
server_key_iv: "ec5b2a0cda8520b5a1026b8ad30f4e86"
server_key_version: "encryptionkey@0"
I20260806 15:03:04.688051 2055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:04.712582 2055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:04.714653 2055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:04.714781 2055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:04.714990 2055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:04.715260 2055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:04.715292 2055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:04.715319 2055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:04.715343 2055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:04.722389 2055 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:44713
I20260806 15:03:04.722447 2253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:44713 every 8 connection(s)
I20260806 15:03:04.722711 2055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
I20260806 15:03:04.727249 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 2055
I20260806 15:03:04.727332 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-1/wal/instance
I20260806 15:03:04.727478 945 external_mini_cluster.cc:1428] Setting key 4a72aa9091b0c7bc93f69e164ce3093f
Aug 06 15:03:04 dist-test-slave-tc2s krb5kdc[1710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028584, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:04.732112 2256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:04.723334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:46513 (local address 127.0.236.66:45329)
0806 15:03:04.723517 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:04.723550 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:04.724010 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:04.724226 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:04.724230 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:04.724767 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:04.724956 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:04.724960 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.725626 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.725627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:04.726466 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:04.726469 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.726841 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.726842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:04.726967 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:04.727402 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:04.727412 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:04.729001 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:04.730515 (+ 1514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:04.730522 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:04.730524 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:04.730807 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:04.731033 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:04.731034 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:04.731034 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:04.731189 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:04.731510 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:04.731516 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:04.731667 (+ 151us) client_negotiation.cc:769] Sending connection context
0806 15:03:04.731761 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 15:03:04.731909 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":110,"threads_started":1}
I20260806 15:03:04.732451 1830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:04.723355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:45329 (local address 127.0.236.126:46513)
0806 15:03:04.723400 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:04.723402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:04.723622 (+ 220us) server_negotiation.cc:407] Connection header received
0806 15:03:04.724099 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:04.724100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:04.724113 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:04.724163 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:04.725046 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.725546 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.726555 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.726774 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.726828 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:04.729109 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:04.729111 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:04.729111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:04.729119 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:04.730434 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:04.730900 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:04.730901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:04.730902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:04.730965 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:04.731252 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:04.731253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:04.731254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:04.731404 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:04.731479 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:04.732256 (+ 777us) server_negotiation.cc:299] Negotiation successful
0806 15:03:04.732350 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:03:04.732831 2254 heartbeater.cc:347] Connected to a master server at 127.0.236.126:46513
I20260806 15:03:04.732928 2254 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:04.733091 2254 heartbeater.cc:511] Master 127.0.236.126:46513 requested a full tablet report, sending...
I20260806 15:03:04.733458 1771 ts_manager.cc:194] Registered new tserver with Master: 6d41695a35a949b8bad075cd8b883be1 (127.0.236.66:44713)
I20260806 15:03:04.733824 1771 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.66@KRBTEST.COM'} at 127.0.236.66:45329
WARNING: no policy specified for kudu/127.0.236.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
I20260806 15:03:04.751658 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:0
--local_ip_for_outbound_sockets=127.0.236.67
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:46513
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:37377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:04.829469 2259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:04.829653 2259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:04.829674 2259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:04.830996 2259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:04.831041 2259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:04.831076 2259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:03:04.832479 2259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:37377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:46513
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:04.832692 2259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:04.832881 2259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:04.837404 2269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:04.837432 2259 server_base.cc:1034] running on GCE node
W20260806 15:03:04.837383 2265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:04.837393 2267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:04.837811 2259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:04.838063 2259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:04.839196 2259 hybrid_clock.cc:648] HybridClock initialized: now 1786028584839184 us; error 29 us; skew 500 ppm
Aug 06 15:03:04 dist-test-slave-tc2s krb5kdc[1710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028584, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:04.841460 2259 init.cc:377] Logged in from keytab as kudu/127.0.236.67@KRBTEST.COM (short username kudu)
I20260806 15:03:04.841842 2259 webserver.cc:460] Webserver started at http://127.0.236.67:40935/ using document root <none> and password file <none>
I20260806 15:03:04.842056 2259 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:04.842104 2259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:04.842228 2259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:04.843161 2259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-2/data/instance:
uuid: "628da071c1704a31b345794c89ef97df"
format_stamp: "Formatted at 2026-08-06 15:03:04 on dist-test-slave-tc2s"
server_key: "77785837511dac169b7d84c08440d92e"
server_key_iv: "4585f75e00b33a4ba084b74aca2048b4"
server_key_version: "encryptionkey@0"
I20260806 15:03:04.843490 2259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-2/wal/instance:
uuid: "628da071c1704a31b345794c89ef97df"
format_stamp: "Formatted at 2026-08-06 15:03:04 on dist-test-slave-tc2s"
server_key: "77785837511dac169b7d84c08440d92e"
server_key_iv: "4585f75e00b33a4ba084b74aca2048b4"
server_key_version: "encryptionkey@0"
I20260806 15:03:04.844781 2259 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:03:04.845461 2282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:04.845606 2259 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:04.845682 2259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "628da071c1704a31b345794c89ef97df"
format_stamp: "Formatted at 2026-08-06 15:03:04 on dist-test-slave-tc2s"
server_key: "77785837511dac169b7d84c08440d92e"
server_key_iv: "4585f75e00b33a4ba084b74aca2048b4"
server_key_version: "encryptionkey@0"
I20260806 15:03:04.845732 2259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:04.857609 2259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:04.859524 2259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:04.859627 2259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:04.859782 2259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:04.860054 2259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:04.860085 2259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:04.860105 2259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:04.860129 2259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:04.865545 2259 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:43783
I20260806 15:03:04.865602 2438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:43783 every 8 connection(s)
I20260806 15:03:04.865852 2259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
Aug 06 15:03:04 dist-test-slave-tc2s krb5kdc[1710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028584, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:04.873612 1830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:04.866498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:34011 (local address 127.0.236.126:46513)
0806 15:03:04.866550 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:04.866552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:04.866736 (+ 184us) server_negotiation.cc:407] Connection header received
0806 15:03:04.867164 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:04.867165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:04.867174 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:04.867223 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:04.868036 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.868525 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.869418 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.869591 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.869632 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:04.871465 (+ 1833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:04.871467 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:04.871467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:04.871473 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:04.872657 (+ 1184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:04.872953 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:04.872953 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:04.872954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:04.872999 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:04.873161 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:04.873162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:04.873162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:04.873251 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:04.873294 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:04.873494 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 15:03:04.873550 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 15:03:04.873773 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:04.866461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:46513 (local address 127.0.236.67:34011)
0806 15:03:04.866628 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:04.866667 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:04.867094 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:04.867271 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:04.867274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:04.867783 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:04.867963 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:04.867967 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.868582 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.868582 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:04.869353 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:04.869355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.869641 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.869642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:04.869731 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:04.870133 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:04.870140 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:04.871390 (+ 1250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:04.872715 (+ 1325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:04.872717 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:04.872719 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:04.872904 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:04.873041 (+ 137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:04.873041 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:04.873041 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:04.873121 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:04.873290 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:04.873293 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:04.873391 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 15:03:04.873479 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 15:03:04.873588 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
I20260806 15:03:04.874213 2440 heartbeater.cc:347] Connected to a master server at 127.0.236.126:46513
I20260806 15:03:04.874294 2440 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:04.874457 2440 heartbeater.cc:511] Master 127.0.236.126:46513 requested a full tablet report, sending...
I20260806 15:03:04.874784 1771 ts_manager.cc:194] Registered new tserver with Master: 628da071c1704a31b345794c89ef97df (127.0.236.67:43783)
I20260806 15:03:04.875121 1771 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.67@KRBTEST.COM'} at 127.0.236.67:34011
I20260806 15:03:04.875794 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 2259
I20260806 15:03:04.875888 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028580791223-945-0/minicluster-data/ts-2/wal/instance
I20260806 15:03:04.876042 945 external_mini_cluster.cc:1428] Setting key 5d52721d7b37863cb157aeeaae6af304
I20260806 15:03:04.876823 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:03:04.884055 1830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:04.878709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45852 (local address 127.0.236.126:46513)
0806 15:03:04.878755 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:04.878757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:04.878859 (+ 102us) server_negotiation.cc:407] Connection header received
0806 15:03:04.878899 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:04.878899 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:04.878907 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:04.878949 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:04.879429 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.879877 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.880631 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.880801 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.880847 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:04.881873 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:04.881874 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:04.881875 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:04.881880 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:04.883050 (+ 1170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:04.883347 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:04.883349 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:04.883349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:04.883397 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:04.883570 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:04.883571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:04.883571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:04.883697 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:04.883780 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:04.883904 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 15:03:04.883974 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 15:03:04.886287 1771 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45852:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:03:04.888091 1771 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:03:04.894999 2451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:04.890802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:44713 (local address 127.0.0.1:52036)
0806 15:03:04.891097 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:04.891133 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:04.891193 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:04.891530 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:04.891531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:04.891533 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:04.891589 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:04.891591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.892702 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.892702 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:04.894057 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:04.894061 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.894722 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.894723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:04.894804 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:04.894817 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 15:03:04.894878 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 15:03:04.894928 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":82,"thread_start_us":54,"threads_started":1}
I20260806 15:03:04.895084 2456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:04.891064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57562 (local address 127.0.236.67:43783)
0806 15:03:04.891225 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:04.891229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:04.891268 (+ 39us) server_negotiation.cc:407] Connection header received
0806 15:03:04.891326 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:04.891328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:04.891338 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:04.891393 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:04.891566 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.892576 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.894124 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.894783 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.894850 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:04.894902 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:04.895003 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 15:03:04.895030 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":76,"threads_started":1}
I20260806 15:03:04.895246 2453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:04.890897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52036 (local address 127.0.236.66:44713)
0806 15:03:04.891381 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:04.891385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:04.891395 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:03:04.891430 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:04.891432 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:04.891447 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:04.891526 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:04.891708 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.892595 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.894162 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.894646 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.894709 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:04.894766 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:04.895120 (+ 354us) server_negotiation.cc:299] Negotiation successful
0806 15:03:04.895170 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":109,"threads_started":1}
I20260806 15:03:04.895398 2452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:04.890798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:34579 (local address 127.0.0.1:42516)
0806 15:03:04.891092 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:04.891131 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:04.891193 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:04.891929 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:04.891930 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:04.891932 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:04.891968 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:04.891969 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.893064 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.893065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:04.893982 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:04.893985 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.895187 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.895187 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:04.895245 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:04.895249 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:03:04.895305 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 15:03:04.895341 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":53,"threads_started":1}
I20260806 15:03:04.895493 2454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:04.890984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:43783 (local address 127.0.0.1:57562)
0806 15:03:04.891217 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:04.891226 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:04.891275 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:04.891435 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:04.891437 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:04.891439 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:04.891501 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:04.891503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.892679 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.892680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:04.894059 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:04.894061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.894862 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.894863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:04.894937 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:04.894940 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:03:04.894982 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 15:03:04.895014 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":71,"thread_start_us":36,"threads_started":1}
I20260806 15:03:04.895531 2455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:04.890969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42516 (local address 127.0.236.65:34579)
0806 15:03:04.891661 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:04.891665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:04.891673 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:04.891748 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:04.891750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:04.891766 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:04.891838 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:04.892047 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.892977 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.894062 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.894783 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.894853 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:04.894903 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:04.895399 (+ 496us) server_negotiation.cc:299] Negotiation successful
0806 15:03:04.895456 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":71,"threads_started":1}
I20260806 15:03:04.896095 2157 tablet_service.cc:1467] Processing CreateTablet for tablet 5bccb30231224198baccaedda4354eab (DEFAULT_TABLE table=test-table [id=3f838a8a48b44e3ab16af2b81beb0cf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:04.896185 1948 tablet_service.cc:1467] Processing CreateTablet for tablet 5bccb30231224198baccaedda4354eab (DEFAULT_TABLE table=test-table [id=3f838a8a48b44e3ab16af2b81beb0cf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:04.896307 2351 tablet_service.cc:1467] Processing CreateTablet for tablet 5bccb30231224198baccaedda4354eab (DEFAULT_TABLE table=test-table [id=3f838a8a48b44e3ab16af2b81beb0cf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:04.896440 2157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bccb30231224198baccaedda4354eab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:04.896538 1948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bccb30231224198baccaedda4354eab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:04.896585 2351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bccb30231224198baccaedda4354eab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:04.899717 2458 tablet_bootstrap.cc:492] T 5bccb30231224198baccaedda4354eab P b49685f1bc644ef4abf906abd2a28670: Bootstrap starting.
I20260806 15:03:04.899770 2457 tablet_bootstrap.cc:492] T 5bccb30231224198baccaedda4354eab P 6d41695a35a949b8bad075cd8b883be1: Bootstrap starting.
I20260806 15:03:04.900295 2457 tablet_bootstrap.cc:654] T 5bccb30231224198baccaedda4354eab P 6d41695a35a949b8bad075cd8b883be1: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:04.900372 2458 tablet_bootstrap.cc:654] T 5bccb30231224198baccaedda4354eab P b49685f1bc644ef4abf906abd2a28670: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:04.900839 2458 log.cc:824] T 5bccb30231224198baccaedda4354eab P b49685f1bc644ef4abf906abd2a28670: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:04.900843 2457 log.cc:824] T 5bccb30231224198baccaedda4354eab P 6d41695a35a949b8bad075cd8b883be1: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:04.901131 2459 tablet_bootstrap.cc:492] T 5bccb30231224198baccaedda4354eab P 628da071c1704a31b345794c89ef97df: Bootstrap starting.
I20260806 15:03:04.901559 2458 tablet_bootstrap.cc:492] T 5bccb30231224198baccaedda4354eab P b49685f1bc644ef4abf906abd2a28670: No bootstrap required, opened a new log
I20260806 15:03:04.901620 2458 ts_tablet_manager.cc:1397] T 5bccb30231224198baccaedda4354eab P b49685f1bc644ef4abf906abd2a28670: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:03:04.901657 2459 tablet_bootstrap.cc:654] T 5bccb30231224198baccaedda4354eab P 628da071c1704a31b345794c89ef97df: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:04.901784 2457 tablet_bootstrap.cc:492] T 5bccb30231224198baccaedda4354eab P 6d41695a35a949b8bad075cd8b883be1: No bootstrap required, opened a new log
I20260806 15:03:04.901852 2457 ts_tablet_manager.cc:1397] T 5bccb30231224198baccaedda4354eab P 6d41695a35a949b8bad075cd8b883be1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:03:04.902207 2459 log.cc:824] T 5bccb30231224198baccaedda4354eab P 628da071c1704a31b345794c89ef97df: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:04.902923 2459 tablet_bootstrap.cc:492] T 5bccb30231224198baccaedda4354eab P 628da071c1704a31b345794c89ef97df: No bootstrap required, opened a new log
I20260806 15:03:04.902993 2459 ts_tablet_manager.cc:1397] T 5bccb30231224198baccaedda4354eab P 628da071c1704a31b345794c89ef97df: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:03:04.903352 2458 raft_consensus.cc:348] T 5bccb30231224198baccaedda4354eab P b49685f1bc644ef4abf906abd2a28670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d41695a35a949b8bad075cd8b883be1" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44713 } } peers { permanent_uuid: "628da071c1704a31b345794c89ef97df" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 43783 } } peers { permanent_uuid: "b49685f1bc644ef4abf906abd2a28670" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34579 } }
I20260806 15:03:04.903506 2458 raft_consensus.cc:374] T 5bccb30231224198baccaedda4354eab P b49685f1bc644ef4abf906abd2a28670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:04.903541 2458 raft_consensus.cc:729] T 5bccb30231224198baccaedda4354eab P b49685f1bc644ef4abf906abd2a28670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b49685f1bc644ef4abf906abd2a28670, State: Initialized, Role: FOLLOWER
I20260806 15:03:04.903672 2458 consensus_queue.cc:260] T 5bccb30231224198baccaedda4354eab P b49685f1bc644ef4abf906abd2a28670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d41695a35a949b8bad075cd8b883be1" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44713 } } peers { permanent_uuid: "628da071c1704a31b345794c89ef97df" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 43783 } } peers { permanent_uuid: "b49685f1bc644ef4abf906abd2a28670" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34579 } }
I20260806 15:03:04.903914 2458 ts_tablet_manager.cc:1428] T 5bccb30231224198baccaedda4354eab P b49685f1bc644ef4abf906abd2a28670: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 15:03:04.903841 2457 raft_consensus.cc:348] T 5bccb30231224198baccaedda4354eab P 6d41695a35a949b8bad075cd8b883be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d41695a35a949b8bad075cd8b883be1" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44713 } } peers { permanent_uuid: "628da071c1704a31b345794c89ef97df" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 43783 } } peers { permanent_uuid: "b49685f1bc644ef4abf906abd2a28670" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34579 } }
I20260806 15:03:04.904013 2457 raft_consensus.cc:374] T 5bccb30231224198baccaedda4354eab P 6d41695a35a949b8bad075cd8b883be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:04.904055 2457 raft_consensus.cc:729] T 5bccb30231224198baccaedda4354eab P 6d41695a35a949b8bad075cd8b883be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d41695a35a949b8bad075cd8b883be1, State: Initialized, Role: FOLLOWER
I20260806 15:03:04.904070 2049 heartbeater.cc:503] Master 127.0.236.126:46513 was elected leader, sending a full tablet report...
I20260806 15:03:04.904165 2457 consensus_queue.cc:260] T 5bccb30231224198baccaedda4354eab P 6d41695a35a949b8bad075cd8b883be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d41695a35a949b8bad075cd8b883be1" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44713 } } peers { permanent_uuid: "628da071c1704a31b345794c89ef97df" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 43783 } } peers { permanent_uuid: "b49685f1bc644ef4abf906abd2a28670" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34579 } }
I20260806 15:03:04.904430 2457 ts_tablet_manager.cc:1428] T 5bccb30231224198baccaedda4354eab P 6d41695a35a949b8bad075cd8b883be1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:03:04.904457 2254 heartbeater.cc:503] Master 127.0.236.126:46513 was elected leader, sending a full tablet report...
I20260806 15:03:04.904764 2459 raft_consensus.cc:348] T 5bccb30231224198baccaedda4354eab P 628da071c1704a31b345794c89ef97df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d41695a35a949b8bad075cd8b883be1" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44713 } } peers { permanent_uuid: "628da071c1704a31b345794c89ef97df" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 43783 } } peers { permanent_uuid: "b49685f1bc644ef4abf906abd2a28670" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34579 } }
I20260806 15:03:04.904920 2459 raft_consensus.cc:374] T 5bccb30231224198baccaedda4354eab P 628da071c1704a31b345794c89ef97df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:04.904959 2459 raft_consensus.cc:729] T 5bccb30231224198baccaedda4354eab P 628da071c1704a31b345794c89ef97df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 628da071c1704a31b345794c89ef97df, State: Initialized, Role: FOLLOWER
I20260806 15:03:04.905081 2459 consensus_queue.cc:260] T 5bccb30231224198baccaedda4354eab P 628da071c1704a31b345794c89ef97df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d41695a35a949b8bad075cd8b883be1" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44713 } } peers { permanent_uuid: "628da071c1704a31b345794c89ef97df" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 43783 } } peers { permanent_uuid: "b49685f1bc644ef4abf906abd2a28670" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34579 } }
I20260806 15:03:04.905342 2459 ts_tablet_manager.cc:1428] T 5bccb30231224198baccaedda4354eab P 628da071c1704a31b345794c89ef97df: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 15:03:04.905478 2440 heartbeater.cc:503] Master 127.0.236.126:46513 was elected leader, sending a full tablet report...
I20260806 15:03:04.935391 2464 raft_consensus.cc:482] T 5bccb30231224198baccaedda4354eab P 6d41695a35a949b8bad075cd8b883be1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:03:04.935492 2464 raft_consensus.cc:504] T 5bccb30231224198baccaedda4354eab P 6d41695a35a949b8bad075cd8b883be1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d41695a35a949b8bad075cd8b883be1" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44713 } } peers { permanent_uuid: "628da071c1704a31b345794c89ef97df" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 43783 } } peers { permanent_uuid: "b49685f1bc644ef4abf906abd2a28670" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34579 } }
I20260806 15:03:04.935977 2464 leader_election.cc:288] T 5bccb30231224198baccaedda4354eab P 6d41695a35a949b8bad075cd8b883be1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 628da071c1704a31b345794c89ef97df (127.0.236.67:43783), b49685f1bc644ef4abf906abd2a28670 (127.0.236.65:34579)
I20260806 15:03:04.939438 2456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:04.936199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:59221 (local address 127.0.236.67:43783)
0806 15:03:04.936236 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:04.936238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:04.936488 (+ 250us) server_negotiation.cc:407] Connection header received
0806 15:03:04.936520 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:04.936521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:04.936530 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:04.936581 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:04.936736 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.937365 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.938601 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.939141 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.939200 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:04.939207 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:04.939340 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 15:03:04.939374 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 15:03:04.939441 2455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:04.936181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:34877 (local address 127.0.236.65:34579)
0806 15:03:04.936226 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:04.936228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:04.936239 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:03:04.936262 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:04.936263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:04.936274 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:04.936327 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:04.936514 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.937199 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.938552 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.939115 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.939190 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:04.939200 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:04.939340 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 15:03:04.939374 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 15:03:04.939463 2256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:04.936131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:34579 (local address 127.0.236.66:34877)
0806 15:03:04.936171 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:04.936181 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:04.936218 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:04.936343 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:04.936344 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:04.936346 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:04.936423 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:04.936425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.937267 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.937268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:04.938475 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:04.938477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.939190 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.939191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:04.939274 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:04.939276 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 15:03:04.939329 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 15:03:04.939371 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":36}
I20260806 15:03:04.939463 2466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:04.936131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:43783 (local address 127.0.236.66:59221)
0806 15:03:04.936411 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:04.936421 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:04.936482 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:04.936582 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:04.936582 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:04.936583 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:04.936672 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:04.936674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.937422 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.937423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:04.938538 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:04.938540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.939205 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.939206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:04.939271 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:04.939275 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:03:04.939329 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 15:03:04.939370 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":132,"thread_start_us":90,"threads_started":1}
I20260806 15:03:04.939834 1986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bccb30231224198baccaedda4354eab" candidate_uuid: "6d41695a35a949b8bad075cd8b883be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b49685f1bc644ef4abf906abd2a28670" is_pre_election: true
I20260806 15:03:04.940009 1986 raft_consensus.cc:2454] T 5bccb30231224198baccaedda4354eab P b49685f1bc644ef4abf906abd2a28670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d41695a35a949b8bad075cd8b883be1 in term 0.
I20260806 15:03:04.939999 2378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bccb30231224198baccaedda4354eab" candidate_uuid: "6d41695a35a949b8bad075cd8b883be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "628da071c1704a31b345794c89ef97df" is_pre_election: true
I20260806 15:03:04.940124 2378 raft_consensus.cc:2454] T 5bccb30231224198baccaedda4354eab P 628da071c1704a31b345794c89ef97df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d41695a35a949b8bad075cd8b883be1 in term 0.
I20260806 15:03:04.940287 2080 leader_election.cc:302] T 5bccb30231224198baccaedda4354eab P 6d41695a35a949b8bad075cd8b883be1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d41695a35a949b8bad075cd8b883be1, b49685f1bc644ef4abf906abd2a28670; no voters:
I20260806 15:03:04.940423 2464 raft_consensus.cc:2781] T 5bccb30231224198baccaedda4354eab P 6d41695a35a949b8bad075cd8b883be1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:03:04.940495 2464 raft_consensus.cc:482] T 5bccb30231224198baccaedda4354eab P 6d41695a35a949b8bad075cd8b883be1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:03:04.940510 2464 raft_consensus.cc:3037] T 5bccb30231224198baccaedda4354eab P 6d41695a35a949b8bad075cd8b883be1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:04.941139 2464 raft_consensus.cc:504] T 5bccb30231224198baccaedda4354eab P 6d41695a35a949b8bad075cd8b883be1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d41695a35a949b8bad075cd8b883be1" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44713 } } peers { permanent_uuid: "628da071c1704a31b345794c89ef97df" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 43783 } } peers { permanent_uuid: "b49685f1bc644ef4abf906abd2a28670" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34579 } }
I20260806 15:03:04.941250 2464 leader_election.cc:288] T 5bccb30231224198baccaedda4354eab P 6d41695a35a949b8bad075cd8b883be1 [CANDIDATE]: Term 1 election: Requested vote from peers 628da071c1704a31b345794c89ef97df (127.0.236.67:43783), b49685f1bc644ef4abf906abd2a28670 (127.0.236.65:34579)
I20260806 15:03:04.941361 2378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bccb30231224198baccaedda4354eab" candidate_uuid: "6d41695a35a949b8bad075cd8b883be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "628da071c1704a31b345794c89ef97df"
I20260806 15:03:04.941421 1986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bccb30231224198baccaedda4354eab" candidate_uuid: "6d41695a35a949b8bad075cd8b883be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b49685f1bc644ef4abf906abd2a28670"
I20260806 15:03:04.941430 2378 raft_consensus.cc:3037] T 5bccb30231224198baccaedda4354eab P 628da071c1704a31b345794c89ef97df [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:04.941491 1986 raft_consensus.cc:3037] T 5bccb30231224198baccaedda4354eab P b49685f1bc644ef4abf906abd2a28670 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:04.942149 2378 raft_consensus.cc:2454] T 5bccb30231224198baccaedda4354eab P 628da071c1704a31b345794c89ef97df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d41695a35a949b8bad075cd8b883be1 in term 1.
I20260806 15:03:04.942152 1986 raft_consensus.cc:2454] T 5bccb30231224198baccaedda4354eab P b49685f1bc644ef4abf906abd2a28670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d41695a35a949b8bad075cd8b883be1 in term 1.
I20260806 15:03:04.942322 2082 leader_election.cc:302] T 5bccb30231224198baccaedda4354eab P 6d41695a35a949b8bad075cd8b883be1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 628da071c1704a31b345794c89ef97df, 6d41695a35a949b8bad075cd8b883be1; no voters:
I20260806 15:03:04.942406 2464 raft_consensus.cc:2781] T 5bccb30231224198baccaedda4354eab P 6d41695a35a949b8bad075cd8b883be1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:04.942556 2464 raft_consensus.cc:686] T 5bccb30231224198baccaedda4354eab P 6d41695a35a949b8bad075cd8b883be1 [term 1 LEADER]: Becoming Leader. State: Replica: 6d41695a35a949b8bad075cd8b883be1, State: Running, Role: LEADER
I20260806 15:03:04.942637 2464 consensus_queue.cc:237] T 5bccb30231224198baccaedda4354eab P 6d41695a35a949b8bad075cd8b883be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d41695a35a949b8bad075cd8b883be1" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44713 } } peers { permanent_uuid: "628da071c1704a31b345794c89ef97df" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 43783 } } peers { permanent_uuid: "b49685f1bc644ef4abf906abd2a28670" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34579 } }
I20260806 15:03:04.943450 1771 catalog_manager.cc:5495] T 5bccb30231224198baccaedda4354eab P 6d41695a35a949b8bad075cd8b883be1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d41695a35a949b8bad075cd8b883be1 (127.0.236.66). New cstate: current_term: 1 leader_uuid: "6d41695a35a949b8bad075cd8b883be1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d41695a35a949b8bad075cd8b883be1" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "628da071c1704a31b345794c89ef97df" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 43783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b49685f1bc644ef4abf906abd2a28670" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34579 } health_report { overall_health: UNKNOWN } } }
Aug 06 15:03:04 dist-test-slave-tc2s krb5kdc[1710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.65@KRBTEST.COM
I20260806 15:03:04.954849 2455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:04.947921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42524 (local address 127.0.236.65:34579)
0806 15:03:04.947970 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:04.947972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:04.948066 (+ 94us) server_negotiation.cc:407] Connection header received
0806 15:03:04.948144 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:04.948145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:04.948154 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:04.948198 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:04.948703 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.949202 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.949878 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.950112 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.950165 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:04.952339 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:04.952347 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:04.952347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:04.952354 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:04.953761 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:04.954144 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:04.954147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:04.954147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:04.954201 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:04.954363 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:04.954364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:04.954364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:04.954491 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:04.954625 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:04.954694 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 15:03:04.954767 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
Aug 06 15:03:04 dist-test-slave-tc2s krb5kdc[1710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028584, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260806 15:03:04.973204 2255 tablet.cc:2369] T 5bccb30231224198baccaedda4354eab P 6d41695a35a949b8bad075cd8b883be1: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 15:03:04 dist-test-slave-tc2s krb5kdc[1710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028584, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.236.65@KRBTEST.COM
I20260806 15:03:04.976969 2455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:04.971223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42536 (local address 127.0.236.65:34579)
0806 15:03:04.971281 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:04.971283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:04.971334 (+ 51us) server_negotiation.cc:407] Connection header received
0806 15:03:04.971373 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:04.971374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:04.971382 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:04.971425 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:04.971888 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.972330 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.972961 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.973162 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.973227 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:04.975143 (+ 1916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:04.975144 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:04.975144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:04.975150 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:04.976150 (+ 1000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:04.976407 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:04.976407 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:04.976408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:04.976450 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:04.976614 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:04.976614 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:04.976614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:04.976689 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:04.976754 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:04.976851 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 15:03:04.976902 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
Aug 06 15:03:04 dist-test-slave-tc2s krb5kdc[1710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 15:03:04 dist-test-slave-tc2s krb5kdc[1710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.65@KRBTEST.COM
I20260806 15:03:04.999441 2455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:04.993482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42544 (local address 127.0.236.65:34579)
0806 15:03:04.993526 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:04.993528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:04.993622 (+ 94us) server_negotiation.cc:407] Connection header received
0806 15:03:04.993684 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:04.993685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:04.993693 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:04.993731 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:04.994248 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.994676 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.995346 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:04.995520 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:04.995572 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:04.997500 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:04.997501 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:04.997501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:04.997507 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:04.998707 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:04.998965 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:04.998965 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:04.998965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:04.999008 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:04.999126 (+ 118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:04.999126 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:04.999126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:04.999198 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:04.999258 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:04.999335 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 15:03:04.999381 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260806 15:03:05.000005 1917 tablet_service.cc:2908] Rejecting scan request for tablet 5bccb30231224198baccaedda4354eab: Uninitialized: clean time has not yet been initialized
I20260806 15:03:05.000794 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 1861
I20260806 15:03:05.020087 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 2055
I20260806 15:03:05.032250 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 2259
I20260806 15:03:05.042870 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 1730
2026-08-06T15:03:05Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2020 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:03:05 dist-test-slave-tc2s krb5kdc[2490](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:03:05 dist-test-slave-tc2s krb5kdc[2490](info): set up 2 sockets
Aug 06 15:03:05 dist-test-slave-tc2s krb5kdc[2490](info): commencing operation
krb5kdc: starting...
W20260806 15:03:06.148590 945 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:03:06 dist-test-slave-tc2s krb5kdc[2490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:03:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:03:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:03:06.256772 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:39061
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:36589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:39061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:06.348119 2508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:06.348302 2508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:06.348323 2508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:06.349391 2508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:03:06.349424 2508 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 15:03:06.349444 2508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:06.349458 2508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:06.349472 2508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:03:06.349486 2508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:03:06.350960 2508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:36589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:39061
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:39061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:06.351147 2508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:06.351305 2508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:06.355765 2515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:06.355845 2519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:06.355764 2516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:06.355887 2508 server_base.cc:1034] running on GCE node
I20260806 15:03:06.356117 2508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:06.356361 2508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:06.357494 2508 hybrid_clock.cc:648] HybridClock initialized: now 1786028586357481 us; error 30 us; skew 500 ppm
Aug 06 15:03:06 dist-test-slave-tc2s krb5kdc[2490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028586, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:06.359719 2508 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:03:06.360103 2508 webserver.cc:460] Webserver started at http://127.0.236.126:39341/ using document root <none> and password file <none>
I20260806 15:03:06.360281 2508 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:06.360317 2508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:06.360419 2508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:06.361320 2508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "b7e616f8dd9b4ba19a577c0faa18bfa7"
format_stamp: "Formatted at 2026-08-06 15:03:06 on dist-test-slave-tc2s"
server_key: "4352dedbe26549896e41f1e5255f0952"
server_key_iv: "861f8a3921f94433c7e438ae967f632b"
server_key_version: "encryptionkey@0"
I20260806 15:03:06.361590 2508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "b7e616f8dd9b4ba19a577c0faa18bfa7"
format_stamp: "Formatted at 2026-08-06 15:03:06 on dist-test-slave-tc2s"
server_key: "4352dedbe26549896e41f1e5255f0952"
server_key_iv: "861f8a3921f94433c7e438ae967f632b"
server_key_version: "encryptionkey@0"
I20260806 15:03:06.362809 2508 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:03:06.363420 2531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:06.363561 2508 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:03:06.363641 2508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "b7e616f8dd9b4ba19a577c0faa18bfa7"
format_stamp: "Formatted at 2026-08-06 15:03:06 on dist-test-slave-tc2s"
server_key: "4352dedbe26549896e41f1e5255f0952"
server_key_iv: "861f8a3921f94433c7e438ae967f632b"
server_key_version: "encryptionkey@0"
I20260806 15:03:06.363696 2508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:06.372325 2508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:06.373968 2508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:06.374089 2508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:06.378587 2508 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:39061
I20260806 15:03:06.378618 2620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:39061 every 8 connection(s)
I20260806 15:03:06.378899 2508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:03:06.379482 2621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:06.381480 2621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7e616f8dd9b4ba19a577c0faa18bfa7: Bootstrap starting.
I20260806 15:03:06.382107 2621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7e616f8dd9b4ba19a577c0faa18bfa7: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:06.382422 2621 log.cc:824] T 00000000000000000000000000000000 P b7e616f8dd9b4ba19a577c0faa18bfa7: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:06.382426 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 2508
I20260806 15:03:06.382519 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:03:06.382679 945 external_mini_cluster.cc:1428] Setting key 6978f4f1c84f63a3446bdbcf0f752378
I20260806 15:03:06.383364 2621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7e616f8dd9b4ba19a577c0faa18bfa7: No bootstrap required, opened a new log
I20260806 15:03:06.385010 2621 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7e616f8dd9b4ba19a577c0faa18bfa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e616f8dd9b4ba19a577c0faa18bfa7" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39061 } }
I20260806 15:03:06.385133 2621 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7e616f8dd9b4ba19a577c0faa18bfa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:06.385152 2621 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7e616f8dd9b4ba19a577c0faa18bfa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7e616f8dd9b4ba19a577c0faa18bfa7, State: Initialized, Role: FOLLOWER
I20260806 15:03:06.385251 2621 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7e616f8dd9b4ba19a577c0faa18bfa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e616f8dd9b4ba19a577c0faa18bfa7" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39061 } }
I20260806 15:03:06.385314 2621 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7e616f8dd9b4ba19a577c0faa18bfa7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:03:06.385336 2621 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7e616f8dd9b4ba19a577c0faa18bfa7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:03:06.385380 2621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7e616f8dd9b4ba19a577c0faa18bfa7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:06.385849 2621 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7e616f8dd9b4ba19a577c0faa18bfa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e616f8dd9b4ba19a577c0faa18bfa7" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39061 } }
I20260806 15:03:06.385964 2621 leader_election.cc:302] T 00000000000000000000000000000000 P b7e616f8dd9b4ba19a577c0faa18bfa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7e616f8dd9b4ba19a577c0faa18bfa7; no voters:
I20260806 15:03:06.386104 2621 leader_election.cc:288] T 00000000000000000000000000000000 P b7e616f8dd9b4ba19a577c0faa18bfa7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:03:06.386133 2629 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7e616f8dd9b4ba19a577c0faa18bfa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:06.386278 2629 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7e616f8dd9b4ba19a577c0faa18bfa7 [term 1 LEADER]: Becoming Leader. State: Replica: b7e616f8dd9b4ba19a577c0faa18bfa7, State: Running, Role: LEADER
I20260806 15:03:06.386404 2621 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7e616f8dd9b4ba19a577c0faa18bfa7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:03:06.386409 2629 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7e616f8dd9b4ba19a577c0faa18bfa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e616f8dd9b4ba19a577c0faa18bfa7" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39061 } }
I20260806 15:03:06.386838 2632 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7e616f8dd9b4ba19a577c0faa18bfa7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7e616f8dd9b4ba19a577c0faa18bfa7. Latest consensus state: current_term: 1 leader_uuid: "b7e616f8dd9b4ba19a577c0faa18bfa7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e616f8dd9b4ba19a577c0faa18bfa7" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39061 } } }
I20260806 15:03:06.386929 2632 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7e616f8dd9b4ba19a577c0faa18bfa7 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:06.387136 2631 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7e616f8dd9b4ba19a577c0faa18bfa7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7e616f8dd9b4ba19a577c0faa18bfa7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e616f8dd9b4ba19a577c0faa18bfa7" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39061 } } }
I20260806 15:03:06.387411 2631 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7e616f8dd9b4ba19a577c0faa18bfa7 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:06.387547 2638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:03:06.388100 2638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:03:06.389356 2638 catalog_manager.cc:1347] Generated new cluster ID: a55f2fb6047945098603b43f537d4169
I20260806 15:03:06.389408 2638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 15:03:06 dist-test-slave-tc2s krb5kdc[2490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:06.393749 2628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:06.383063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40852 (local address 127.0.236.126:39061)
0806 15:03:06.383197 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:06.383204 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:06.383227 (+ 23us) server_negotiation.cc:407] Connection header received
0806 15:03:06.383642 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:06.383649 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:06.383703 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:06.383810 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:06.384526 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.386049 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.387019 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.387605 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.387688 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:06.390505 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:06.390509 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:06.390509 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:06.390520 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:06.392258 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:06.392653 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:06.392655 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:06.392655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:06.392747 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:06.392943 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:06.392944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:06.392944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:06.393202 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:06.393337 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:06.393450 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 15:03:06.393569 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":58,"threads_started":1}
I20260806 15:03:06.402732 2638 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:03:06.403244 2638 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:03:06.406666 2638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7e616f8dd9b4ba19a577c0faa18bfa7: Generated new TSK 0
I20260806 15:03:06.406810 2638 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.236.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
I20260806 15:03:06.433543 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:0
--local_ip_for_outbound_sockets=127.0.236.65
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:39061
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:36589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:06.513701 2659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:06.513927 2659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:06.513967 2659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:06.515067 2659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:06.515100 2659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:06.515118 2659 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 15:03:06.515139 2659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:03:06.516578 2659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:36589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.236.126:39061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:06.516762 2659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:06.516932 2659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:06.521538 2668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:06.521538 2666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:06.521602 2665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:06.521564 2659 server_base.cc:1034] running on GCE node
I20260806 15:03:06.521961 2659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:06.522189 2659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:06.523334 2659 hybrid_clock.cc:648] HybridClock initialized: now 1786028586523321 us; error 24 us; skew 500 ppm
Aug 06 15:03:06 dist-test-slave-tc2s krb5kdc[2490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028586, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:06.525695 2659 init.cc:377] Logged in from keytab as kudu/127.0.236.65@KRBTEST.COM (short username kudu)
I20260806 15:03:06.526180 2659 webserver.cc:460] Webserver started at http://127.0.236.65:39315/ using document root <none> and password file <none>
I20260806 15:03:06.526394 2659 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:06.526443 2659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:06.526566 2659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:06.527556 2659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-0/data/instance:
uuid: "f103800ea1744a2fa4e7e3ac01bee388"
format_stamp: "Formatted at 2026-08-06 15:03:06 on dist-test-slave-tc2s"
server_key: "7631b714c6a1c655d0c950bbfaf3d971"
server_key_iv: "e30a9c1420fcdde35cd95fae92c9e73f"
server_key_version: "encryptionkey@0"
I20260806 15:03:06.527873 2659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-0/wal/instance:
uuid: "f103800ea1744a2fa4e7e3ac01bee388"
format_stamp: "Formatted at 2026-08-06 15:03:06 on dist-test-slave-tc2s"
server_key: "7631b714c6a1c655d0c950bbfaf3d971"
server_key_iv: "e30a9c1420fcdde35cd95fae92c9e73f"
server_key_version: "encryptionkey@0"
I20260806 15:03:06.529037 2659 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 15:03:06.529626 2679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:06.529772 2659 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:06.529861 2659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "f103800ea1744a2fa4e7e3ac01bee388"
format_stamp: "Formatted at 2026-08-06 15:03:06 on dist-test-slave-tc2s"
server_key: "7631b714c6a1c655d0c950bbfaf3d971"
server_key_iv: "e30a9c1420fcdde35cd95fae92c9e73f"
server_key_version: "encryptionkey@0"
I20260806 15:03:06.529955 2659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:06.546025 2659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:06.547580 2659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:06.547717 2659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:06.548183 2659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:06.548223 2659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:06.548247 2659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:06.548260 2659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 15:03:06 dist-test-slave-tc2s krb5kdc[2490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028586, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:06.554939 2659 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:39611
I20260806 15:03:06.555239 2659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:03:06.555241 2836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:39611 every 8 connection(s)
I20260806 15:03:06.557363 2628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:06.548670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:34837 (local address 127.0.236.126:39061)
0806 15:03:06.548719 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:06.548721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:06.548932 (+ 211us) server_negotiation.cc:407] Connection header received
0806 15:03:06.549414 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:06.549417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:06.549432 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:06.549500 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:06.550172 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.550697 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.551658 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.551861 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.551916 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:06.554745 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:06.554746 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:06.554747 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:06.554756 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:06.556139 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:06.556497 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:06.556498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:06.556498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:06.556554 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:06.556774 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:06.556775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:06.556776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:06.556891 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:06.556956 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:06.557193 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 15:03:06.557274 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 15:03:06.557520 2695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:06.548666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39061 (local address 127.0.236.65:34837)
0806 15:03:06.548823 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:06.548856 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:06.549295 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:06.549565 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:06.549569 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:06.549828 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:06.550084 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:06.550089 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.550780 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.550781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:06.551575 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:06.551578 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.551930 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.551931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:06.552074 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:06.552583 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:06.552592 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:06.554653 (+ 2061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:06.556211 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:06.556217 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:06.556219 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:06.556423 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:06.556606 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:06.556607 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:06.556608 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:06.556717 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:06.556946 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:06.556953 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:06.557081 (+ 128us) client_negotiation.cc:769] Sending connection context
0806 15:03:06.557165 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 15:03:06.557310 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
I20260806 15:03:06.557825 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 2659
I20260806 15:03:06.557945 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-0/wal/instance
I20260806 15:03:06.558065 2838 heartbeater.cc:347] Connected to a master server at 127.0.236.126:39061
I20260806 15:03:06.558110 945 external_mini_cluster.cc:1428] Setting key 5c1b9d3eec8bec7ffae37a91d0d9f35b
I20260806 15:03:06.558157 2838 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:06.558471 2838 heartbeater.cc:511] Master 127.0.236.126:39061 requested a full tablet report, sending...
I20260806 15:03:06.558980 2561 ts_manager.cc:194] Registered new tserver with Master: f103800ea1744a2fa4e7e3ac01bee388 (127.0.236.65:39611)
I20260806 15:03:06.559726 2561 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.65@KRBTEST.COM'} at 127.0.236.65:34837
I20260806 15:03:06.565266 2844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:06.560176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39061 (local address 127.0.236.65:39223)
0806 15:03:06.560290 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:06.560300 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:06.560407 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:06.560563 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:06.560564 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:06.560732 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:06.560774 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:06.560776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.561391 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.561392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:06.561888 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:06.561912 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.562280 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.562281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:06.562339 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:06.562682 (+ 343us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:06.562686 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:06.562922 (+ 236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:06.564408 (+ 1486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:06.564408 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:06.564409 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:06.564567 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:06.564744 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:06.564745 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:06.564745 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:06.564789 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:06.565023 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:06.565023 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:06.565057 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 15:03:06.565121 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 15:03:06.565207 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":66,"threads_started":1}
I20260806 15:03:06.565315 2628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:06.560183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:39223 (local address 127.0.236.126:39061)
0806 15:03:06.560220 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:06.560223 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:06.560381 (+ 158us) server_negotiation.cc:407] Connection header received
0806 15:03:06.560471 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:06.560472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:06.560481 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:06.560508 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:06.560853 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.561299 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.561972 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.562203 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.562263 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:06.563010 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:06.563011 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:06.563012 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:06.563018 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:06.564323 (+ 1305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:06.564629 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:06.564630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:06.564630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:06.564690 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:06.564841 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:06.564842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:06.564842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:06.564967 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:06.565030 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:06.565161 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 15:03:06.565230 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for kudu/127.0.236.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
I20260806 15:03:06.582149 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:0
--local_ip_for_outbound_sockets=127.0.236.66
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:39061
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:36589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:06.660640 2848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:06.660811 2848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:06.660833 2848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:06.661952 2848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:06.661991 2848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:06.662012 2848 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 15:03:06.662045 2848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:03:06.663501 2848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:36589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.236.126:39061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:06.663702 2848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:06.663893 2848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 15:03:06.668599 2848 server_base.cc:1034] running on GCE node
W20260806 15:03:06.668543 2858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:06.668689 2857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:06.668545 2862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:06.668931 2848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:06.669148 2848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:06.670284 2848 hybrid_clock.cc:648] HybridClock initialized: now 1786028586670269 us; error 27 us; skew 500 ppm
Aug 06 15:03:06 dist-test-slave-tc2s krb5kdc[2490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028586, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:06.672514 2848 init.cc:377] Logged in from keytab as kudu/127.0.236.66@KRBTEST.COM (short username kudu)
I20260806 15:03:06.672895 2848 webserver.cc:460] Webserver started at http://127.0.236.66:45867/ using document root <none> and password file <none>
I20260806 15:03:06.673089 2848 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:06.673126 2848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:06.673213 2848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:06.674160 2848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-1/data/instance:
uuid: "10f59698d1e543faab4809d8ccd21971"
format_stamp: "Formatted at 2026-08-06 15:03:06 on dist-test-slave-tc2s"
server_key: "8e07d048b2044614e365f7c4f57a3887"
server_key_iv: "206c3dfa75e734e4e8435a6d3e7f713d"
server_key_version: "encryptionkey@0"
I20260806 15:03:06.674425 2848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-1/wal/instance:
uuid: "10f59698d1e543faab4809d8ccd21971"
format_stamp: "Formatted at 2026-08-06 15:03:06 on dist-test-slave-tc2s"
server_key: "8e07d048b2044614e365f7c4f57a3887"
server_key_iv: "206c3dfa75e734e4e8435a6d3e7f713d"
server_key_version: "encryptionkey@0"
I20260806 15:03:06.675513 2848 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:06.676115 2873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:06.676261 2848 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:06.676334 2848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "10f59698d1e543faab4809d8ccd21971"
format_stamp: "Formatted at 2026-08-06 15:03:06 on dist-test-slave-tc2s"
server_key: "8e07d048b2044614e365f7c4f57a3887"
server_key_iv: "206c3dfa75e734e4e8435a6d3e7f713d"
server_key_version: "encryptionkey@0"
I20260806 15:03:06.676398 2848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:06.688521 2848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:06.690168 2848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:06.690289 2848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:06.690745 2848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:06.690784 2848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:06.690804 2848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:06.690816 2848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:06.696585 2848 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:42819
I20260806 15:03:06.696612 3045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:42819 every 8 connection(s)
Aug 06 15:03:06 dist-test-slave-tc2s krb5kdc[2490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028586, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:06.696955 2848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
I20260806 15:03:06.699694 2628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:06.691137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:54093 (local address 127.0.236.126:39061)
0806 15:03:06.691206 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:06.691208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:06.691338 (+ 130us) server_negotiation.cc:407] Connection header received
0806 15:03:06.691736 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:06.691737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:06.691750 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:06.691813 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:06.692557 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.693295 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.694749 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.695031 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.695107 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:06.697462 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:06.697464 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:06.697464 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:06.697470 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:06.698723 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:06.699046 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:06.699047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:06.699047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:06.699090 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:06.699257 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:06.699258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:06.699258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:06.699348 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:06.699389 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:06.699581 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 15:03:06.699633 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 15:03:06.699831 2888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:06.691077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39061 (local address 127.0.236.66:54093)
0806 15:03:06.691225 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:06.691261 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:06.691655 (+ 394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:06.691875 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:06.691879 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:06.692191 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:06.692456 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:06.692463 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.693383 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.693384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:06.694646 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:06.694649 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.695164 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.695165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:06.695262 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:06.695687 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:06.695697 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:06.697375 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:06.698789 (+ 1414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:06.698793 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:06.698794 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:06.698995 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:06.699133 (+ 138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:06.699134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:06.699134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:06.699217 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:06.699387 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:06.699390 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:06.699498 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 15:03:06.699568 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 15:03:06.699675 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
I20260806 15:03:06.700294 3047 heartbeater.cc:347] Connected to a master server at 127.0.236.126:39061
I20260806 15:03:06.700379 3047 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:06.700529 3047 heartbeater.cc:511] Master 127.0.236.126:39061 requested a full tablet report, sending...
I20260806 15:03:06.700947 2560 ts_manager.cc:194] Registered new tserver with Master: 10f59698d1e543faab4809d8ccd21971 (127.0.236.66:42819)
I20260806 15:03:06.701427 2560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.66@KRBTEST.COM'} at 127.0.236.66:54093
I20260806 15:03:06.705756 3055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:06.701164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39061 (local address 127.0.236.66:40885)
0806 15:03:06.701281 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:06.701291 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:06.701345 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:06.701456 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:06.701456 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:06.701636 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:06.701701 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:06.701702 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.702481 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.702482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:06.702969 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:06.702970 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.703230 (+ 260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.703231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:06.703273 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:06.703574 (+ 301us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:06.703577 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:06.703810 (+ 233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:06.705070 (+ 1260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:06.705071 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:06.705072 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:06.705200 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:06.705319 (+ 119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:06.705320 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:06.705320 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:06.705358 (+ 38us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:06.705555 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:06.705555 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:06.705587 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 15:03:06.705627 (+ 40us) client_negotiation.cc:240] Negotiation successful
0806 15:03:06.705691 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":67,"threads_started":1}
I20260806 15:03:06.705757 2628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:06.701243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:40885 (local address 127.0.236.126:39061)
0806 15:03:06.701280 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:06.701282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:06.701333 (+ 51us) server_negotiation.cc:407] Connection header received
0806 15:03:06.701393 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:06.701393 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:06.701400 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:06.701444 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:06.701771 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.702398 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.703024 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.703187 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.703225 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:06.703872 (+ 647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:06.703873 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:06.703874 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:06.703878 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:06.705015 (+ 1137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:06.705246 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:06.705246 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:06.705247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:06.705286 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:06.705388 (+ 102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:06.705389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:06.705389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:06.705512 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:06.705556 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:06.705650 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 15:03:06.705704 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 15:03:06.706310 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 2848
I20260806 15:03:06.706400 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-1/wal/instance
I20260806 15:03:06.706570 945 external_mini_cluster.cc:1428] Setting key a42dfa62982e6c3ec94fddeedf5012ad
WARNING: no policy specified for kudu/127.0.236.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
I20260806 15:03:06.728904 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:0
--local_ip_for_outbound_sockets=127.0.236.67
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:39061
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:36589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:06.807175 3058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:06.807361 3058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:06.807394 3058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:06.808491 3058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:06.808530 3058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:06.808557 3058 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 15:03:06.808580 3058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:03:06.810047 3058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:36589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.236.126:39061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:06.810261 3058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:06.810465 3058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:06.815006 3064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:06.815009 3063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:06.815011 3067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:06.815191 3058 server_base.cc:1034] running on GCE node
I20260806 15:03:06.815378 3058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:06.815601 3058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:06.816748 3058 hybrid_clock.cc:648] HybridClock initialized: now 1786028586816735 us; error 27 us; skew 500 ppm
Aug 06 15:03:06 dist-test-slave-tc2s krb5kdc[2490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028586, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:06.819100 3058 init.cc:377] Logged in from keytab as kudu/127.0.236.67@KRBTEST.COM (short username kudu)
I20260806 15:03:06.819515 3058 webserver.cc:460] Webserver started at http://127.0.236.67:36421/ using document root <none> and password file <none>
I20260806 15:03:06.819703 3058 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:06.819751 3058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:06.819877 3058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:06.820839 3058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-2/data/instance:
uuid: "cf8789339ccc43b98ef3f890059006d3"
format_stamp: "Formatted at 2026-08-06 15:03:06 on dist-test-slave-tc2s"
server_key: "2808928ec3ff4dada1adddf0f64bdd6b"
server_key_iv: "ea5ce94d71a4e6781ec0cc2eef8e9c32"
server_key_version: "encryptionkey@0"
I20260806 15:03:06.821117 3058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-2/wal/instance:
uuid: "cf8789339ccc43b98ef3f890059006d3"
format_stamp: "Formatted at 2026-08-06 15:03:06 on dist-test-slave-tc2s"
server_key: "2808928ec3ff4dada1adddf0f64bdd6b"
server_key_iv: "ea5ce94d71a4e6781ec0cc2eef8e9c32"
server_key_version: "encryptionkey@0"
I20260806 15:03:06.822284 3058 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:03:06.822898 3078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:06.823050 3058 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:03:06.823125 3058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "cf8789339ccc43b98ef3f890059006d3"
format_stamp: "Formatted at 2026-08-06 15:03:06 on dist-test-slave-tc2s"
server_key: "2808928ec3ff4dada1adddf0f64bdd6b"
server_key_iv: "ea5ce94d71a4e6781ec0cc2eef8e9c32"
server_key_version: "encryptionkey@0"
I20260806 15:03:06.823191 3058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:06.839627 3058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:06.841606 3058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:06.841728 3058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:06.842238 3058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:06.842283 3058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:06.842311 3058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:06.842337 3058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 15:03:06 dist-test-slave-tc2s krb5kdc[2490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028586, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:06.848583 3058 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:37323
I20260806 15:03:06.848682 3251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:37323 every 8 connection(s)
I20260806 15:03:06.848949 3058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
I20260806 15:03:06.851027 2628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:06.842548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:50335 (local address 127.0.236.126:39061)
0806 15:03:06.842597 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:06.842599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:06.842811 (+ 212us) server_negotiation.cc:407] Connection header received
0806 15:03:06.843261 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:06.843263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:06.843276 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:06.843327 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:06.843985 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.844725 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.846107 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.846399 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.846474 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:06.848584 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:06.848587 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:06.848587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:06.848595 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:06.850041 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:06.850379 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:06.850379 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:06.850379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:06.850427 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:06.850590 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:06.850591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:06.850591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:06.850684 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:06.850723 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:06.850901 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 15:03:06.850965 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 15:03:06.851179 3096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:06.842542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39061 (local address 127.0.236.67:50335)
0806 15:03:06.842688 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:06.842732 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:06.843170 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:06.843388 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:06.843392 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:06.843629 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:06.843874 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:06.843880 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.844816 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.844817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:06.846044 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:06.846046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.846486 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.846487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:06.846616 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:06.847057 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:06.847065 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:06.848492 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:06.850117 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:06.850120 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:06.850122 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:06.850324 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:06.850465 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:06.850466 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:06.850466 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:06.850550 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:06.850720 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:06.850723 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:06.850821 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 15:03:06.850890 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 15:03:06.851005 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":68,"threads_started":1}
I20260806 15:03:06.851601 3252 heartbeater.cc:347] Connected to a master server at 127.0.236.126:39061
I20260806 15:03:06.851677 3252 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:06.851837 3252 heartbeater.cc:511] Master 127.0.236.126:39061 requested a full tablet report, sending...
I20260806 15:03:06.852203 2561 ts_manager.cc:194] Registered new tserver with Master: cf8789339ccc43b98ef3f890059006d3 (127.0.236.67:37323)
I20260806 15:03:06.852650 2561 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.67@KRBTEST.COM'} at 127.0.236.67:50335
I20260806 15:03:06.852927 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 3058
I20260806 15:03:06.853012 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-2/wal/instance
I20260806 15:03:06.853166 945 external_mini_cluster.cc:1428] Setting key 0222b8a4e9d567878b87f7dadc61f741
I20260806 15:03:06.854522 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:03:06.857939 3258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:06.852480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39061 (local address 127.0.236.67:49595)
0806 15:03:06.852595 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:06.852606 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:06.852649 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:06.852890 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:06.852891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:06.853090 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:06.853143 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:06.853144 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.853992 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.853993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:06.854643 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:06.854645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.855060 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.855060 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:06.855103 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:06.855458 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:06.855463 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:06.855709 (+ 246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:06.857035 (+ 1326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:06.857036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:06.857037 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:06.857192 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:06.857354 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:06.857355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:06.857355 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:06.857420 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:06.857640 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:06.857641 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:06.857684 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 15:03:06.857751 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 15:03:06.857836 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":63,"threads_started":1}
I20260806 15:03:06.857962 2628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:06.852538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:49595 (local address 127.0.236.126:39061)
0806 15:03:06.852783 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:06.852785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:06.852794 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:03:06.852811 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:06.852812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:06.852821 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:06.852871 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:06.853210 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.853881 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.854720 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.855001 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.855062 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:06.855784 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:06.855785 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:06.855785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:06.855791 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:06.856966 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:06.857250 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:06.857251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:06.857252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:06.857303 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:06.857487 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:06.857488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:06.857488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:06.857586 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:06.857638 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:06.857801 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 15:03:06.857862 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210}
Aug 06 15:03:06 dist-test-slave-tc2s krb5kdc[2490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028586, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 15:03:06 dist-test-slave-tc2s krb5kdc[2490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028586, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:06.876427 2628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:06.870870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40860 (local address 127.0.236.126:39061)
0806 15:03:06.870915 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:06.870917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:06.870925 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:06.870961 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:06.870961 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:06.870968 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:06.871013 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:06.871486 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.871926 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.872545 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.872706 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.872745 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:06.874625 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:06.874627 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:06.874627 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:06.874633 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:06.875645 (+ 1012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:06.875929 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:06.875930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:06.875930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:06.875971 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:06.876093 (+ 122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:06.876094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:06.876094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:06.876179 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:06.876242 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:06.876311 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 15:03:06.876368 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 15:03:06.877717 2561 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:03:06.879352 2561 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:03:06.884972 3268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:06.881273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:42819 (local address 127.0.0.1:56296)
0806 15:03:06.881486 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:06.881496 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:06.881548 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:06.881769 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:06.881772 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:06.881774 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:06.881833 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:06.881834 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.882767 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.882768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:06.883781 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:06.883784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.884682 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.884683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:06.884764 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:06.884777 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 15:03:06.884839 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 15:03:06.884891 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":69,"threads_started":1}
I20260806 15:03:06.884992 3275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:06.881460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56296 (local address 127.0.236.66:42819)
0806 15:03:06.881623 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:06.881628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:06.881637 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:03:06.881670 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:06.881672 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:06.881683 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:06.881746 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:06.881911 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.882692 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.883898 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.884612 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.884670 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:06.884744 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:06.884863 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 15:03:06.884900 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":70,"threads_started":1}
I20260806 15:03:06.885318 3273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:06.881391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:39611 (local address 127.0.0.1:51406)
0806 15:03:06.881779 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:06.881788 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:06.881839 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:06.882041 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:06.882042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:06.882044 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:06.882095 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:06.882096 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.883403 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.883404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:06.884239 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:06.884241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.885115 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.885117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:06.885180 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:06.885185 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 15:03:06.885213 (+ 28us) client_negotiation.cc:240] Negotiation successful
0806 15:03:06.885250 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":82,"threads_started":1}
I20260806 15:03:06.885658 3270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:06.881200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:37323 (local address 127.0.0.1:34008)
0806 15:03:06.881398 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:06.881438 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:06.881496 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:06.882123 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:06.882124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:06.882126 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:06.882156 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:06.882157 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.883374 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.883375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:06.884413 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:06.884416 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.885468 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.885469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:06.885530 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:06.885533 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:03:06.885560 (+ 27us) client_negotiation.cc:240] Negotiation successful
0806 15:03:06.885595 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":61,"threads_started":1}
I20260806 15:03:06.885676 3277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:06.881614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51406 (local address 127.0.236.65:39611)
0806 15:03:06.881767 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:06.881772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:06.881829 (+ 57us) server_negotiation.cc:407] Connection header received
0806 15:03:06.881928 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:06.881930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:06.881946 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:06.882167 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:06.882229 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.883306 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.884312 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.885009 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.885428 (+ 419us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:06.885520 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:06.885544 (+ 24us) server_negotiation.cc:299] Negotiation successful
0806 15:03:06.885584 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":72,"threads_started":1}
I20260806 15:03:06.885699 2959 tablet_service.cc:1467] Processing CreateTablet for tablet 2a94e5f4d3fe4b1db55afccc5ea05174 (DEFAULT_TABLE table=test-table [id=85dfa6ac6eca4faa974243b20769456d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:06.886114 2959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a94e5f4d3fe4b1db55afccc5ea05174. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:06.886235 3272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:06.881295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34008 (local address 127.0.236.67:37323)
0806 15:03:06.881976 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:06.881979 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:06.881987 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:06.882021 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:06.882023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:06.882035 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:06.882218 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:06.882263 (+ 45us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.883291 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.884497 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.885306 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.886024 (+ 718us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:06.886092 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:06.886117 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 15:03:06.886151 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":70,"threads_started":1}
I20260806 15:03:06.886344 2754 tablet_service.cc:1467] Processing CreateTablet for tablet 2a94e5f4d3fe4b1db55afccc5ea05174 (DEFAULT_TABLE table=test-table [id=85dfa6ac6eca4faa974243b20769456d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:06.886735 2754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a94e5f4d3fe4b1db55afccc5ea05174. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:06.886780 3158 tablet_service.cc:1467] Processing CreateTablet for tablet 2a94e5f4d3fe4b1db55afccc5ea05174 (DEFAULT_TABLE table=test-table [id=85dfa6ac6eca4faa974243b20769456d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:06.887133 3158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a94e5f4d3fe4b1db55afccc5ea05174. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:06.890058 3281 tablet_bootstrap.cc:492] T 2a94e5f4d3fe4b1db55afccc5ea05174 P cf8789339ccc43b98ef3f890059006d3: Bootstrap starting.
I20260806 15:03:06.890214 3283 tablet_bootstrap.cc:492] T 2a94e5f4d3fe4b1db55afccc5ea05174 P 10f59698d1e543faab4809d8ccd21971: Bootstrap starting.
I20260806 15:03:06.890601 3284 tablet_bootstrap.cc:492] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388: Bootstrap starting.
I20260806 15:03:06.890802 3281 tablet_bootstrap.cc:654] T 2a94e5f4d3fe4b1db55afccc5ea05174 P cf8789339ccc43b98ef3f890059006d3: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:06.890801 3283 tablet_bootstrap.cc:654] T 2a94e5f4d3fe4b1db55afccc5ea05174 P 10f59698d1e543faab4809d8ccd21971: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:06.891130 3281 log.cc:824] T 2a94e5f4d3fe4b1db55afccc5ea05174 P cf8789339ccc43b98ef3f890059006d3: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:06.891130 3283 log.cc:824] T 2a94e5f4d3fe4b1db55afccc5ea05174 P 10f59698d1e543faab4809d8ccd21971: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:06.891163 3284 tablet_bootstrap.cc:654] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:06.891467 3284 log.cc:824] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:06.891865 3283 tablet_bootstrap.cc:492] T 2a94e5f4d3fe4b1db55afccc5ea05174 P 10f59698d1e543faab4809d8ccd21971: No bootstrap required, opened a new log
I20260806 15:03:06.891947 3283 ts_tablet_manager.cc:1397] T 2a94e5f4d3fe4b1db55afccc5ea05174 P 10f59698d1e543faab4809d8ccd21971: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:03:06.892063 3281 tablet_bootstrap.cc:492] T 2a94e5f4d3fe4b1db55afccc5ea05174 P cf8789339ccc43b98ef3f890059006d3: No bootstrap required, opened a new log
I20260806 15:03:06.892134 3281 ts_tablet_manager.cc:1397] T 2a94e5f4d3fe4b1db55afccc5ea05174 P cf8789339ccc43b98ef3f890059006d3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:03:06.892217 3284 tablet_bootstrap.cc:492] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388: No bootstrap required, opened a new log
I20260806 15:03:06.892280 3284 ts_tablet_manager.cc:1397] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:03:06.893488 3283 raft_consensus.cc:348] T 2a94e5f4d3fe4b1db55afccc5ea05174 P 10f59698d1e543faab4809d8ccd21971 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } } peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } }
I20260806 15:03:06.893616 3283 raft_consensus.cc:374] T 2a94e5f4d3fe4b1db55afccc5ea05174 P 10f59698d1e543faab4809d8ccd21971 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:06.893647 3283 raft_consensus.cc:729] T 2a94e5f4d3fe4b1db55afccc5ea05174 P 10f59698d1e543faab4809d8ccd21971 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10f59698d1e543faab4809d8ccd21971, State: Initialized, Role: FOLLOWER
I20260806 15:03:06.893772 3283 consensus_queue.cc:260] T 2a94e5f4d3fe4b1db55afccc5ea05174 P 10f59698d1e543faab4809d8ccd21971 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } } peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } }
I20260806 15:03:06.894052 3283 ts_tablet_manager.cc:1428] T 2a94e5f4d3fe4b1db55afccc5ea05174 P 10f59698d1e543faab4809d8ccd21971: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:03:06.894167 3047 heartbeater.cc:503] Master 127.0.236.126:39061 was elected leader, sending a full tablet report...
I20260806 15:03:06.894080 3281 raft_consensus.cc:348] T 2a94e5f4d3fe4b1db55afccc5ea05174 P cf8789339ccc43b98ef3f890059006d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } } peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } }
I20260806 15:03:06.894224 3281 raft_consensus.cc:374] T 2a94e5f4d3fe4b1db55afccc5ea05174 P cf8789339ccc43b98ef3f890059006d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:06.894151 3284 raft_consensus.cc:348] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } } peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } }
I20260806 15:03:06.894268 3284 raft_consensus.cc:374] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:06.894265 3281 raft_consensus.cc:729] T 2a94e5f4d3fe4b1db55afccc5ea05174 P cf8789339ccc43b98ef3f890059006d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf8789339ccc43b98ef3f890059006d3, State: Initialized, Role: FOLLOWER
I20260806 15:03:06.894307 3284 raft_consensus.cc:729] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f103800ea1744a2fa4e7e3ac01bee388, State: Initialized, Role: FOLLOWER
I20260806 15:03:06.894377 3281 consensus_queue.cc:260] T 2a94e5f4d3fe4b1db55afccc5ea05174 P cf8789339ccc43b98ef3f890059006d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } } peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } }
I20260806 15:03:06.894423 3284 consensus_queue.cc:260] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } } peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } }
I20260806 15:03:06.894650 3284 ts_tablet_manager.cc:1428] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 15:03:06.894650 3281 ts_tablet_manager.cc:1428] T 2a94e5f4d3fe4b1db55afccc5ea05174 P cf8789339ccc43b98ef3f890059006d3: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 15:03:06.894771 2838 heartbeater.cc:503] Master 127.0.236.126:39061 was elected leader, sending a full tablet report...
I20260806 15:03:06.894809 3252 heartbeater.cc:503] Master 127.0.236.126:39061 was elected leader, sending a full tablet report...
I20260806 15:03:06.913064 3291 raft_consensus.cc:482] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:03:06.913143 3291 raft_consensus.cc:504] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } } peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } }
I20260806 15:03:06.913378 3291 leader_election.cc:288] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf8789339ccc43b98ef3f890059006d3 (127.0.236.67:37323), 10f59698d1e543faab4809d8ccd21971 (127.0.236.66:42819)
I20260806 15:03:06.915932 3275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:06.913652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:52575 (local address 127.0.236.66:42819)
0806 15:03:06.913688 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:06.913689 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:06.913728 (+ 39us) server_negotiation.cc:407] Connection header received
0806 15:03:06.913785 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:06.913786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:06.913794 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:06.913850 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:06.913996 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.914417 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.915301 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.915640 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.915702 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:06.915709 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:06.915839 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 15:03:06.915872 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:03:06.915961 3272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:06.913563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:51915 (local address 127.0.236.67:37323)
0806 15:03:06.913603 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:06.913604 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:06.913614 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:03:06.913684 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:06.913686 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:06.913695 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:06.913748 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:06.913879 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.914380 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.915302 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.915637 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.915693 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:06.915701 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:06.915870 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 15:03:06.915902 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 15:03:06.915963 3292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:06.913522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:42819 (local address 127.0.236.65:52575)
0806 15:03:06.913674 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:06.913683 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:06.913741 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:06.913852 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:06.913852 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:06.913854 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:06.913944 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:06.913945 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.914502 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.914502 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:06.915216 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:06.915218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.915712 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.915713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:06.915779 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:06.915782 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:03:06.915820 (+ 38us) client_negotiation.cc:240] Negotiation successful
0806 15:03:06.915853 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":10,"thread_start_us":90,"threads_started":1}
I20260806 15:03:06.915995 2695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:06.913516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:37323 (local address 127.0.236.65:51915)
0806 15:03:06.913566 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:06.913578 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:06.913620 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:06.913757 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:06.913758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:06.913759 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:06.913821 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:06.913822 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.914451 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.914452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:06.915213 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:06.915216 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.915708 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.915709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:06.915800 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:06.915803 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:03:06.915854 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 15:03:06.915888 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 15:03:06.916273 2990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a94e5f4d3fe4b1db55afccc5ea05174" candidate_uuid: "f103800ea1744a2fa4e7e3ac01bee388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10f59698d1e543faab4809d8ccd21971" is_pre_election: true
I20260806 15:03:06.916339 3184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a94e5f4d3fe4b1db55afccc5ea05174" candidate_uuid: "f103800ea1744a2fa4e7e3ac01bee388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf8789339ccc43b98ef3f890059006d3" is_pre_election: true
I20260806 15:03:06.916396 2990 raft_consensus.cc:2454] T 2a94e5f4d3fe4b1db55afccc5ea05174 P 10f59698d1e543faab4809d8ccd21971 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f103800ea1744a2fa4e7e3ac01bee388 in term 0.
I20260806 15:03:06.916456 3184 raft_consensus.cc:2454] T 2a94e5f4d3fe4b1db55afccc5ea05174 P cf8789339ccc43b98ef3f890059006d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f103800ea1744a2fa4e7e3ac01bee388 in term 0.
I20260806 15:03:06.916621 2681 leader_election.cc:302] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10f59698d1e543faab4809d8ccd21971, f103800ea1744a2fa4e7e3ac01bee388; no voters:
I20260806 15:03:06.916739 3291 raft_consensus.cc:2781] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:03:06.916792 3291 raft_consensus.cc:482] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:03:06.916808 3291 raft_consensus.cc:3037] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:06.917685 3291 raft_consensus.cc:504] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } } peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } }
I20260806 15:03:06.917794 3291 leader_election.cc:288] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388 [CANDIDATE]: Term 1 election: Requested vote from peers cf8789339ccc43b98ef3f890059006d3 (127.0.236.67:37323), 10f59698d1e543faab4809d8ccd21971 (127.0.236.66:42819)
I20260806 15:03:06.917961 2990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a94e5f4d3fe4b1db55afccc5ea05174" candidate_uuid: "f103800ea1744a2fa4e7e3ac01bee388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10f59698d1e543faab4809d8ccd21971"
I20260806 15:03:06.917954 3184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a94e5f4d3fe4b1db55afccc5ea05174" candidate_uuid: "f103800ea1744a2fa4e7e3ac01bee388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf8789339ccc43b98ef3f890059006d3"
I20260806 15:03:06.918026 3184 raft_consensus.cc:3037] T 2a94e5f4d3fe4b1db55afccc5ea05174 P cf8789339ccc43b98ef3f890059006d3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:06.918026 2990 raft_consensus.cc:3037] T 2a94e5f4d3fe4b1db55afccc5ea05174 P 10f59698d1e543faab4809d8ccd21971 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:06.918780 3184 raft_consensus.cc:2454] T 2a94e5f4d3fe4b1db55afccc5ea05174 P cf8789339ccc43b98ef3f890059006d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f103800ea1744a2fa4e7e3ac01bee388 in term 1.
I20260806 15:03:06.918780 2990 raft_consensus.cc:2454] T 2a94e5f4d3fe4b1db55afccc5ea05174 P 10f59698d1e543faab4809d8ccd21971 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f103800ea1744a2fa4e7e3ac01bee388 in term 1.
I20260806 15:03:06.918926 2681 leader_election.cc:302] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10f59698d1e543faab4809d8ccd21971, f103800ea1744a2fa4e7e3ac01bee388; no voters:
I20260806 15:03:06.919032 3291 raft_consensus.cc:2781] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:06.919144 3291 raft_consensus.cc:686] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388 [term 1 LEADER]: Becoming Leader. State: Replica: f103800ea1744a2fa4e7e3ac01bee388, State: Running, Role: LEADER
I20260806 15:03:06.919221 3291 consensus_queue.cc:237] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } } peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } }
I20260806 15:03:06.919963 2561 catalog_manager.cc:5495] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388 reported cstate change: term changed from 0 to 1, leader changed from <none> to f103800ea1744a2fa4e7e3ac01bee388 (127.0.236.65). New cstate: current_term: 1 leader_uuid: "f103800ea1744a2fa4e7e3ac01bee388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } health_report { overall_health: UNKNOWN } } }
I20260806 15:03:06.934931 3277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:06.932649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51412 (local address 127.0.236.65:39611)
0806 15:03:06.932695 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:06.932697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:06.932739 (+ 42us) server_negotiation.cc:407] Connection header received
0806 15:03:06.932823 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:06.932823 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:06.932839 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:06.932912 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:03:06.933034 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.933459 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.934250 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:06.934431 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:06.934481 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:06.934662 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:03:06.934744 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:03:06.934809 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:06.934853 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 15:03:06.934871 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 15:03:06.936365 3184 raft_consensus.cc:1261] T 2a94e5f4d3fe4b1db55afccc5ea05174 P cf8789339ccc43b98ef3f890059006d3 [term 1 FOLLOWER]: Refusing update from remote peer f103800ea1744a2fa4e7e3ac01bee388: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:06.936688 3293 consensus_queue.cc:1035] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:06.938778 2990 raft_consensus.cc:1261] T 2a94e5f4d3fe4b1db55afccc5ea05174 P 10f59698d1e543faab4809d8ccd21971 [term 1 FOLLOWER]: Refusing update from remote peer f103800ea1744a2fa4e7e3ac01bee388: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:06.939105 3291 consensus_queue.cc:1035] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:06.942564 2561 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:06.942690 2561 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:06.943524 2561 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7e616f8dd9b4ba19a577c0faa18bfa7: Sending DeleteTablet for 3 replicas of tablet 2a94e5f4d3fe4b1db55afccc5ea05174
I20260806 15:03:06.943975 3158 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a94e5f4d3fe4b1db55afccc5ea05174 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:06 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:34008
I20260806 15:03:06.943974 2959 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a94e5f4d3fe4b1db55afccc5ea05174 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:06 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:56296
I20260806 15:03:06.944180 3311 tablet_replica.cc:330] T 2a94e5f4d3fe4b1db55afccc5ea05174 P cf8789339ccc43b98ef3f890059006d3: stopping tablet replica
I20260806 15:03:06.944198 2754 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a94e5f4d3fe4b1db55afccc5ea05174 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:06 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:51406
I20260806 15:03:06.944268 3311 raft_consensus.cc:2229] T 2a94e5f4d3fe4b1db55afccc5ea05174 P cf8789339ccc43b98ef3f890059006d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:06.944380 3311 raft_consensus.cc:2258] T 2a94e5f4d3fe4b1db55afccc5ea05174 P cf8789339ccc43b98ef3f890059006d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:06.944406 3313 tablet_replica.cc:330] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388: stopping tablet replica
I20260806 15:03:06.944581 3313 raft_consensus.cc:2229] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388 [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:03:06.944623 2561 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:03:06.944765 2561 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:03:06.944864 3311 ts_tablet_manager.cc:1905] T 2a94e5f4d3fe4b1db55afccc5ea05174 P cf8789339ccc43b98ef3f890059006d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:06.944973 3313 raft_consensus.cc:2258] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:06.945436 3313 ts_tablet_manager.cc:1905] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:06.946089 3158 tablet_service.cc:1467] Processing CreateTablet for tablet 8a3f703bbb0b407cac538b4c1e7e8357 (DEFAULT_TABLE table=test-table [id=ae17d168a1b94361b0038f4807add155]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:06.946208 3158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a3f703bbb0b407cac538b4c1e7e8357. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:06.946762 3311 ts_tablet_manager.cc:1918] T 2a94e5f4d3fe4b1db55afccc5ea05174 P cf8789339ccc43b98ef3f890059006d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:03:06.946835 3311 log.cc:1196] T 2a94e5f4d3fe4b1db55afccc5ea05174 P cf8789339ccc43b98ef3f890059006d3: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/2a94e5f4d3fe4b1db55afccc5ea05174
I20260806 15:03:06.947100 3311 ts_tablet_manager.cc:1939] T 2a94e5f4d3fe4b1db55afccc5ea05174 P cf8789339ccc43b98ef3f890059006d3: Deleting consensus metadata
I20260806 15:03:06.947229 3313 ts_tablet_manager.cc:1918] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:03:06.947345 3313 log.cc:1196] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-0/wal/wals/2a94e5f4d3fe4b1db55afccc5ea05174
I20260806 15:03:06.947526 3281 tablet_bootstrap.cc:492] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3: Bootstrap starting.
I20260806 15:03:06.947806 3313 ts_tablet_manager.cc:1939] T 2a94e5f4d3fe4b1db55afccc5ea05174 P f103800ea1744a2fa4e7e3ac01bee388: Deleting consensus metadata
I20260806 15:03:06.947971 3281 tablet_bootstrap.cc:654] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:06.948316 2536 catalog_manager.cc:4841] TS cf8789339ccc43b98ef3f890059006d3 (127.0.236.67:37323): tablet 2a94e5f4d3fe4b1db55afccc5ea05174 (table test-table [id=85dfa6ac6eca4faa974243b20769456d]) successfully deleted
I20260806 15:03:06.948505 2536 catalog_manager.cc:4841] TS f103800ea1744a2fa4e7e3ac01bee388 (127.0.236.65:39611): tablet 2a94e5f4d3fe4b1db55afccc5ea05174 (table test-table [id=85dfa6ac6eca4faa974243b20769456d]) successfully deleted
I20260806 15:03:06.948523 3281 tablet_bootstrap.cc:492] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3: No bootstrap required, opened a new log
I20260806 15:03:06.948563 3281 ts_tablet_manager.cc:1397] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:06.948691 3281 raft_consensus.cc:348] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } }
I20260806 15:03:06.948719 2959 tablet_service.cc:1467] Processing CreateTablet for tablet 8a3f703bbb0b407cac538b4c1e7e8357 (DEFAULT_TABLE table=test-table [id=ae17d168a1b94361b0038f4807add155]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:06.948752 3281 raft_consensus.cc:374] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:06.948770 3281 raft_consensus.cc:729] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf8789339ccc43b98ef3f890059006d3, State: Initialized, Role: FOLLOWER
I20260806 15:03:06.948777 2754 tablet_service.cc:1467] Processing CreateTablet for tablet 8a3f703bbb0b407cac538b4c1e7e8357 (DEFAULT_TABLE table=test-table [id=ae17d168a1b94361b0038f4807add155]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:06.948817 2959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a3f703bbb0b407cac538b4c1e7e8357. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:06.948807 3281 consensus_queue.cc:260] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } }
I20260806 15:03:06.948872 2754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a3f703bbb0b407cac538b4c1e7e8357. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:06.948881 3281 ts_tablet_manager.cc:1428] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:06.949704 3283 tablet_bootstrap.cc:492] T 8a3f703bbb0b407cac538b4c1e7e8357 P 10f59698d1e543faab4809d8ccd21971: Bootstrap starting.
I20260806 15:03:06.950050 3284 tablet_bootstrap.cc:492] T 8a3f703bbb0b407cac538b4c1e7e8357 P f103800ea1744a2fa4e7e3ac01bee388: Bootstrap starting.
I20260806 15:03:06.950343 3283 tablet_bootstrap.cc:654] T 8a3f703bbb0b407cac538b4c1e7e8357 P 10f59698d1e543faab4809d8ccd21971: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:06.950453 3284 tablet_bootstrap.cc:654] T 8a3f703bbb0b407cac538b4c1e7e8357 P f103800ea1744a2fa4e7e3ac01bee388: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:06.950888 3283 tablet_bootstrap.cc:492] T 8a3f703bbb0b407cac538b4c1e7e8357 P 10f59698d1e543faab4809d8ccd21971: No bootstrap required, opened a new log
I20260806 15:03:06.950942 3283 ts_tablet_manager.cc:1397] T 8a3f703bbb0b407cac538b4c1e7e8357 P 10f59698d1e543faab4809d8ccd21971: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:06.951112 3283 raft_consensus.cc:348] T 8a3f703bbb0b407cac538b4c1e7e8357 P 10f59698d1e543faab4809d8ccd21971 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } }
I20260806 15:03:06.951177 3283 raft_consensus.cc:374] T 8a3f703bbb0b407cac538b4c1e7e8357 P 10f59698d1e543faab4809d8ccd21971 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:06.951196 3283 raft_consensus.cc:729] T 8a3f703bbb0b407cac538b4c1e7e8357 P 10f59698d1e543faab4809d8ccd21971 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10f59698d1e543faab4809d8ccd21971, State: Initialized, Role: FOLLOWER
I20260806 15:03:06.951234 3283 consensus_queue.cc:260] T 8a3f703bbb0b407cac538b4c1e7e8357 P 10f59698d1e543faab4809d8ccd21971 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } }
I20260806 15:03:06.951248 3312 tablet_replica.cc:330] T 2a94e5f4d3fe4b1db55afccc5ea05174 P 10f59698d1e543faab4809d8ccd21971: stopping tablet replica
I20260806 15:03:06.951319 3283 ts_tablet_manager.cc:1428] T 8a3f703bbb0b407cac538b4c1e7e8357 P 10f59698d1e543faab4809d8ccd21971: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:06.951360 3312 raft_consensus.cc:2229] T 2a94e5f4d3fe4b1db55afccc5ea05174 P 10f59698d1e543faab4809d8ccd21971 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:06.951452 3312 raft_consensus.cc:2258] T 2a94e5f4d3fe4b1db55afccc5ea05174 P 10f59698d1e543faab4809d8ccd21971 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:06.951720 3284 tablet_bootstrap.cc:492] T 8a3f703bbb0b407cac538b4c1e7e8357 P f103800ea1744a2fa4e7e3ac01bee388: No bootstrap required, opened a new log
I20260806 15:03:06.951769 3284 ts_tablet_manager.cc:1397] T 8a3f703bbb0b407cac538b4c1e7e8357 P f103800ea1744a2fa4e7e3ac01bee388: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:03:06.951907 3284 raft_consensus.cc:348] T 8a3f703bbb0b407cac538b4c1e7e8357 P f103800ea1744a2fa4e7e3ac01bee388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } }
I20260806 15:03:06.951987 3284 raft_consensus.cc:374] T 8a3f703bbb0b407cac538b4c1e7e8357 P f103800ea1744a2fa4e7e3ac01bee388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:06.952016 3284 raft_consensus.cc:729] T 8a3f703bbb0b407cac538b4c1e7e8357 P f103800ea1744a2fa4e7e3ac01bee388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f103800ea1744a2fa4e7e3ac01bee388, State: Initialized, Role: FOLLOWER
I20260806 15:03:06.952054 3284 consensus_queue.cc:260] T 8a3f703bbb0b407cac538b4c1e7e8357 P f103800ea1744a2fa4e7e3ac01bee388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } }
I20260806 15:03:06.952137 3284 ts_tablet_manager.cc:1428] T 8a3f703bbb0b407cac538b4c1e7e8357 P f103800ea1744a2fa4e7e3ac01bee388: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:06.952545 3312 ts_tablet_manager.cc:1905] T 2a94e5f4d3fe4b1db55afccc5ea05174 P 10f59698d1e543faab4809d8ccd21971: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:06.954149 3312 ts_tablet_manager.cc:1918] T 2a94e5f4d3fe4b1db55afccc5ea05174 P 10f59698d1e543faab4809d8ccd21971: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:03:06.954217 3312 log.cc:1196] T 2a94e5f4d3fe4b1db55afccc5ea05174 P 10f59698d1e543faab4809d8ccd21971: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-1/wal/wals/2a94e5f4d3fe4b1db55afccc5ea05174
I20260806 15:03:06.954478 3312 ts_tablet_manager.cc:1939] T 2a94e5f4d3fe4b1db55afccc5ea05174 P 10f59698d1e543faab4809d8ccd21971: Deleting consensus metadata
I20260806 15:03:06.954885 2533 catalog_manager.cc:4841] TS 10f59698d1e543faab4809d8ccd21971 (127.0.236.66:42819): tablet 2a94e5f4d3fe4b1db55afccc5ea05174 (table test-table [id=85dfa6ac6eca4faa974243b20769456d]) successfully deleted
W20260806 15:03:07.056103 2839 tablet.cc:2369] T 8a3f703bbb0b407cac538b4c1e7e8357 P f103800ea1744a2fa4e7e3ac01bee388: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 15:03:07.099474 3253 tablet.cc:2369] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:03:07.106419 3290 raft_consensus.cc:482] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:03:07.106489 3290 raft_consensus.cc:504] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } }
I20260806 15:03:07.106786 3290 leader_election.cc:288] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f103800ea1744a2fa4e7e3ac01bee388 (127.0.236.65:39611), 10f59698d1e543faab4809d8ccd21971 (127.0.236.66:42819)
I20260806 15:03:07.109457 3275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:07.107015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:50543 (local address 127.0.236.66:42819)
0806 15:03:07.107073 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:07.107075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:07.107169 (+ 94us) server_negotiation.cc:407] Connection header received
0806 15:03:07.107233 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:07.107234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:07.107243 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:07.107290 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:07.107443 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.107895 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.108807 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.109149 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.109211 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:07.109219 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:07.109354 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 15:03:07.109389 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 15:03:07.109464 3096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:07.106946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:39611 (local address 127.0.236.67:39757)
0806 15:03:07.106995 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:07.107006 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:07.107053 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:07.107204 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:07.107205 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:07.107207 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:07.107284 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:07.107286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.107920 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.107921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:07.108712 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:07.108714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.109216 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.109217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:07.109289 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:07.109292 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:03:07.109336 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 15:03:07.109391 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":30}
I20260806 15:03:07.109460 3277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:07.107012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:39757 (local address 127.0.236.65:39611)
0806 15:03:07.107086 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:07.107088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:07.107097 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:03:07.107125 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:07.107126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:07.107138 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:07.107192 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:07.107351 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.107849 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.108800 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.109144 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.109202 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:07.109211 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:07.109352 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 15:03:07.109386 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 15:03:07.109488 3318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:07.106946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:42819 (local address 127.0.236.67:50543)
0806 15:03:07.107103 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:07.107113 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:07.107171 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:07.107298 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:07.107298 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:07.107300 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:07.107385 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:07.107387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.107968 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.107969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:07.108723 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:07.108725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.109240 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.109241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:07.109291 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:07.109294 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:03:07.109339 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 15:03:07.109392 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":52,"thread_start_us":58,"threads_started":1}
I20260806 15:03:07.109678 2990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a3f703bbb0b407cac538b4c1e7e8357" candidate_uuid: "cf8789339ccc43b98ef3f890059006d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10f59698d1e543faab4809d8ccd21971" is_pre_election: true
I20260806 15:03:07.109764 2990 raft_consensus.cc:2454] T 8a3f703bbb0b407cac538b4c1e7e8357 P 10f59698d1e543faab4809d8ccd21971 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf8789339ccc43b98ef3f890059006d3 in term 0.
I20260806 15:03:07.109704 2782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a3f703bbb0b407cac538b4c1e7e8357" candidate_uuid: "cf8789339ccc43b98ef3f890059006d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f103800ea1744a2fa4e7e3ac01bee388" is_pre_election: true
I20260806 15:03:07.109823 2782 raft_consensus.cc:2454] T 8a3f703bbb0b407cac538b4c1e7e8357 P f103800ea1744a2fa4e7e3ac01bee388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf8789339ccc43b98ef3f890059006d3 in term 0.
I20260806 15:03:07.109941 3081 leader_election.cc:302] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10f59698d1e543faab4809d8ccd21971, cf8789339ccc43b98ef3f890059006d3; no voters:
I20260806 15:03:07.110061 3290 raft_consensus.cc:2781] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:03:07.110101 3290 raft_consensus.cc:482] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:03:07.110126 3290 raft_consensus.cc:3037] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:07.110761 3290 raft_consensus.cc:504] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } }
I20260806 15:03:07.110872 3290 leader_election.cc:288] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3 [CANDIDATE]: Term 1 election: Requested vote from peers f103800ea1744a2fa4e7e3ac01bee388 (127.0.236.65:39611), 10f59698d1e543faab4809d8ccd21971 (127.0.236.66:42819)
I20260806 15:03:07.111019 2782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a3f703bbb0b407cac538b4c1e7e8357" candidate_uuid: "cf8789339ccc43b98ef3f890059006d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f103800ea1744a2fa4e7e3ac01bee388"
I20260806 15:03:07.111081 2782 raft_consensus.cc:3037] T 8a3f703bbb0b407cac538b4c1e7e8357 P f103800ea1744a2fa4e7e3ac01bee388 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:07.111066 2990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a3f703bbb0b407cac538b4c1e7e8357" candidate_uuid: "cf8789339ccc43b98ef3f890059006d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10f59698d1e543faab4809d8ccd21971"
I20260806 15:03:07.111119 2990 raft_consensus.cc:3037] T 8a3f703bbb0b407cac538b4c1e7e8357 P 10f59698d1e543faab4809d8ccd21971 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:07.111677 2990 raft_consensus.cc:2454] T 8a3f703bbb0b407cac538b4c1e7e8357 P 10f59698d1e543faab4809d8ccd21971 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf8789339ccc43b98ef3f890059006d3 in term 1.
I20260806 15:03:07.111721 2782 raft_consensus.cc:2454] T 8a3f703bbb0b407cac538b4c1e7e8357 P f103800ea1744a2fa4e7e3ac01bee388 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf8789339ccc43b98ef3f890059006d3 in term 1.
I20260806 15:03:07.111830 3081 leader_election.cc:302] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10f59698d1e543faab4809d8ccd21971, cf8789339ccc43b98ef3f890059006d3; no voters:
I20260806 15:03:07.111960 3290 raft_consensus.cc:2781] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:07.112082 3290 raft_consensus.cc:686] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3 [term 1 LEADER]: Becoming Leader. State: Replica: cf8789339ccc43b98ef3f890059006d3, State: Running, Role: LEADER
I20260806 15:03:07.112136 3290 consensus_queue.cc:237] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } }
I20260806 15:03:07.112783 2561 catalog_manager.cc:5495] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf8789339ccc43b98ef3f890059006d3 (127.0.236.67). New cstate: current_term: 1 leader_uuid: "cf8789339ccc43b98ef3f890059006d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } health_report { overall_health: HEALTHY } } }
Aug 06 15:03:07 dist-test-slave-tc2s krb5kdc[2490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028586, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:07.152318 3333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:07.146453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39061 (local address 127.0.0.1:40862)
0806 15:03:07.146599 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:07.146609 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:07.146665 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:07.146762 (+ 97us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:07.146763 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:07.147049 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:07.147092 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:07.147094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.147648 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.147649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:07.148155 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:07.148156 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.148408 (+ 252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.148408 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:07.148450 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:07.148915 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:07.148919 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:07.150187 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:07.151454 (+ 1267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:07.151461 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:07.151463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:07.151669 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:07.151827 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:07.151827 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:07.151828 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:07.151880 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:07.152050 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:07.152057 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:07.152097 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 15:03:07.152142 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 15:03:07.152227 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":73,"threads_started":1}
I20260806 15:03:07.152360 2628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:07.146486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40862 (local address 127.0.236.126:39061)
0806 15:03:07.146534 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:07.146537 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:07.146660 (+ 123us) server_negotiation.cc:407] Connection header received
0806 15:03:07.146699 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:07.146700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:07.146708 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:07.146750 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:07.147143 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.147599 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.148203 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.148365 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.148403 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:07.150267 (+ 1864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:07.150268 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:07.150269 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:07.150274 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:07.151384 (+ 1110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:07.151726 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:07.151727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:07.151727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:07.151791 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:07.151923 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:07.151923 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:07.151923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:07.152012 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:07.152054 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:07.152163 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 15:03:07.152271 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":26}
I20260806 15:03:07.154107 2561 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:40862:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 15:03:07.154371 2561 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:03:07.155637 2959 tablet_service.cc:1467] Processing CreateTablet for tablet d7239eebe59243dd914382170ae34e60 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=42bad8da0128471db3e3d27a6f085d0f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 15:03:07.155648 2754 tablet_service.cc:1467] Processing CreateTablet for tablet d7239eebe59243dd914382170ae34e60 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=42bad8da0128471db3e3d27a6f085d0f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 15:03:07.155820 2959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7239eebe59243dd914382170ae34e60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:07.155828 2754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7239eebe59243dd914382170ae34e60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:07.156864 3283 tablet_bootstrap.cc:492] T d7239eebe59243dd914382170ae34e60 P 10f59698d1e543faab4809d8ccd21971: Bootstrap starting.
I20260806 15:03:07.156867 3284 tablet_bootstrap.cc:492] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388: Bootstrap starting.
I20260806 15:03:07.157157 3284 tablet_bootstrap.cc:654] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:07.157176 3283 tablet_bootstrap.cc:654] T d7239eebe59243dd914382170ae34e60 P 10f59698d1e543faab4809d8ccd21971: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:07.155627 3158 tablet_service.cc:1467] Processing CreateTablet for tablet d7239eebe59243dd914382170ae34e60 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=42bad8da0128471db3e3d27a6f085d0f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 15:03:07.157580 3158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7239eebe59243dd914382170ae34e60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:07.157663 3284 tablet_bootstrap.cc:492] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388: No bootstrap required, opened a new log
I20260806 15:03:07.157714 3284 ts_tablet_manager.cc:1397] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260806 15:03:07.157843 3284 raft_consensus.cc:348] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } }
I20260806 15:03:07.157927 3284 raft_consensus.cc:374] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:07.157963 3284 raft_consensus.cc:729] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f103800ea1744a2fa4e7e3ac01bee388, State: Initialized, Role: FOLLOWER
I20260806 15:03:07.158011 3284 consensus_queue.cc:260] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } }
I20260806 15:03:07.158109 3284 ts_tablet_manager.cc:1428] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:07.158131 3291 tablet_replica.cc:439] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } } }
I20260806 15:03:07.158229 3291 tablet_replica.cc:442] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:03:07.158358 3283 tablet_bootstrap.cc:492] T d7239eebe59243dd914382170ae34e60 P 10f59698d1e543faab4809d8ccd21971: No bootstrap required, opened a new log
I20260806 15:03:07.158401 3283 ts_tablet_manager.cc:1397] T d7239eebe59243dd914382170ae34e60 P 10f59698d1e543faab4809d8ccd21971: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:03:07.158557 3283 raft_consensus.cc:348] T d7239eebe59243dd914382170ae34e60 P 10f59698d1e543faab4809d8ccd21971 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } }
I20260806 15:03:07.158618 3283 raft_consensus.cc:374] T d7239eebe59243dd914382170ae34e60 P 10f59698d1e543faab4809d8ccd21971 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:07.158638 3283 raft_consensus.cc:729] T d7239eebe59243dd914382170ae34e60 P 10f59698d1e543faab4809d8ccd21971 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10f59698d1e543faab4809d8ccd21971, State: Initialized, Role: FOLLOWER
I20260806 15:03:07.158677 3283 consensus_queue.cc:260] T d7239eebe59243dd914382170ae34e60 P 10f59698d1e543faab4809d8ccd21971 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } }
I20260806 15:03:07.158763 3283 ts_tablet_manager.cc:1428] T d7239eebe59243dd914382170ae34e60 P 10f59698d1e543faab4809d8ccd21971: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:07.158826 3306 tablet_replica.cc:439] T d7239eebe59243dd914382170ae34e60 P 10f59698d1e543faab4809d8ccd21971: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } } }
I20260806 15:03:07.158917 3306 tablet_replica.cc:442] T d7239eebe59243dd914382170ae34e60 P 10f59698d1e543faab4809d8ccd21971: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:03:07.158994 3281 tablet_bootstrap.cc:492] T d7239eebe59243dd914382170ae34e60 P cf8789339ccc43b98ef3f890059006d3: Bootstrap starting.
I20260806 15:03:07.159384 3281 tablet_bootstrap.cc:654] T d7239eebe59243dd914382170ae34e60 P cf8789339ccc43b98ef3f890059006d3: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:07.160001 3281 tablet_bootstrap.cc:492] T d7239eebe59243dd914382170ae34e60 P cf8789339ccc43b98ef3f890059006d3: No bootstrap required, opened a new log
I20260806 15:03:07.160046 3281 ts_tablet_manager.cc:1397] T d7239eebe59243dd914382170ae34e60 P cf8789339ccc43b98ef3f890059006d3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:07.160185 3281 raft_consensus.cc:348] T d7239eebe59243dd914382170ae34e60 P cf8789339ccc43b98ef3f890059006d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } }
I20260806 15:03:07.160234 3281 raft_consensus.cc:374] T d7239eebe59243dd914382170ae34e60 P cf8789339ccc43b98ef3f890059006d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:07.160256 3281 raft_consensus.cc:729] T d7239eebe59243dd914382170ae34e60 P cf8789339ccc43b98ef3f890059006d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf8789339ccc43b98ef3f890059006d3, State: Initialized, Role: FOLLOWER
I20260806 15:03:07.160301 3281 consensus_queue.cc:260] T d7239eebe59243dd914382170ae34e60 P cf8789339ccc43b98ef3f890059006d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } }
I20260806 15:03:07.160395 3281 ts_tablet_manager.cc:1428] T d7239eebe59243dd914382170ae34e60 P cf8789339ccc43b98ef3f890059006d3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:07.160442 3290 tablet_replica.cc:439] T d7239eebe59243dd914382170ae34e60 P cf8789339ccc43b98ef3f890059006d3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } } }
I20260806 15:03:07.160513 3290 tablet_replica.cc:442] T d7239eebe59243dd914382170ae34e60 P cf8789339ccc43b98ef3f890059006d3: This TxnStatusTablet replica's current role is: FOLLOWER
W20260806 15:03:07.197873 3048 tablet.cc:2369] T 8a3f703bbb0b407cac538b4c1e7e8357 P 10f59698d1e543faab4809d8ccd21971: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:03:07.232237 3291 raft_consensus.cc:482] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:03:07.232302 3291 raft_consensus.cc:504] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } }
I20260806 15:03:07.232435 3291 leader_election.cc:288] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10f59698d1e543faab4809d8ccd21971 (127.0.236.66:42819), cf8789339ccc43b98ef3f890059006d3 (127.0.236.67:37323)
I20260806 15:03:07.232594 3184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7239eebe59243dd914382170ae34e60" candidate_uuid: "f103800ea1744a2fa4e7e3ac01bee388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf8789339ccc43b98ef3f890059006d3" is_pre_election: true
I20260806 15:03:07.232600 2990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7239eebe59243dd914382170ae34e60" candidate_uuid: "f103800ea1744a2fa4e7e3ac01bee388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10f59698d1e543faab4809d8ccd21971" is_pre_election: true
I20260806 15:03:07.232667 2990 raft_consensus.cc:2454] T d7239eebe59243dd914382170ae34e60 P 10f59698d1e543faab4809d8ccd21971 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f103800ea1744a2fa4e7e3ac01bee388 in term 0.
I20260806 15:03:07.232666 3184 raft_consensus.cc:2454] T d7239eebe59243dd914382170ae34e60 P cf8789339ccc43b98ef3f890059006d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f103800ea1744a2fa4e7e3ac01bee388 in term 0.
I20260806 15:03:07.232784 2681 leader_election.cc:302] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10f59698d1e543faab4809d8ccd21971, f103800ea1744a2fa4e7e3ac01bee388; no voters:
I20260806 15:03:07.232901 3291 raft_consensus.cc:2781] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:03:07.232947 3291 raft_consensus.cc:482] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:03:07.232960 3291 raft_consensus.cc:3037] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:07.233533 3291 raft_consensus.cc:504] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } }
I20260806 15:03:07.233646 3291 leader_election.cc:288] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388 [CANDIDATE]: Term 1 election: Requested vote from peers 10f59698d1e543faab4809d8ccd21971 (127.0.236.66:42819), cf8789339ccc43b98ef3f890059006d3 (127.0.236.67:37323)
I20260806 15:03:07.233791 2990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7239eebe59243dd914382170ae34e60" candidate_uuid: "f103800ea1744a2fa4e7e3ac01bee388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10f59698d1e543faab4809d8ccd21971"
I20260806 15:03:07.233799 3184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7239eebe59243dd914382170ae34e60" candidate_uuid: "f103800ea1744a2fa4e7e3ac01bee388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf8789339ccc43b98ef3f890059006d3"
I20260806 15:03:07.233855 2990 raft_consensus.cc:3037] T d7239eebe59243dd914382170ae34e60 P 10f59698d1e543faab4809d8ccd21971 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:07.233855 3184 raft_consensus.cc:3037] T d7239eebe59243dd914382170ae34e60 P cf8789339ccc43b98ef3f890059006d3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:07.234382 2990 raft_consensus.cc:2454] T d7239eebe59243dd914382170ae34e60 P 10f59698d1e543faab4809d8ccd21971 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f103800ea1744a2fa4e7e3ac01bee388 in term 1.
I20260806 15:03:07.234534 2681 leader_election.cc:302] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10f59698d1e543faab4809d8ccd21971, f103800ea1744a2fa4e7e3ac01bee388; no voters:
I20260806 15:03:07.234618 3291 raft_consensus.cc:2781] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:07.234663 3291 raft_consensus.cc:686] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388 [term 1 LEADER]: Becoming Leader. State: Replica: f103800ea1744a2fa4e7e3ac01bee388, State: Running, Role: LEADER
I20260806 15:03:07.234700 3184 raft_consensus.cc:2454] T d7239eebe59243dd914382170ae34e60 P cf8789339ccc43b98ef3f890059006d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f103800ea1744a2fa4e7e3ac01bee388 in term 1.
I20260806 15:03:07.234709 3291 consensus_queue.cc:237] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } }
I20260806 15:03:07.234951 3293 tablet_replica.cc:439] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388: TxnStatusTablet state changed. Reason: New leader f103800ea1744a2fa4e7e3ac01bee388. Latest consensus state: current_term: 1 leader_uuid: "f103800ea1744a2fa4e7e3ac01bee388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } } }
I20260806 15:03:07.235023 3293 tablet_replica.cc:442] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388: This TxnStatusTablet replica's current role is: LEADER
I20260806 15:03:07.235168 3340 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 15:03:07.235262 3340 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 15:03:07.235212 2561 catalog_manager.cc:5495] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388 reported cstate change: term changed from 0 to 1, leader changed from <none> to f103800ea1744a2fa4e7e3ac01bee388 (127.0.236.65). New cstate: current_term: 1 leader_uuid: "f103800ea1744a2fa4e7e3ac01bee388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } health_report { overall_health: UNKNOWN } } }
I20260806 15:03:07.253165 3277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:07.250675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51426 (local address 127.0.236.65:39611)
0806 15:03:07.250722 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:07.250724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:07.250734 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:03:07.250786 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:07.250787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:07.250798 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:07.250851 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:03:07.250943 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.251677 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.252395 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.252697 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.252758 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:07.252964 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:03:07.252996 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:03:07.253036 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:07.253101 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 15:03:07.253119 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 15:03:07.253223 3333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:07.250606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:39611 (local address 127.0.0.1:51426)
0806 15:03:07.250675 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:07.250688 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:07.250735 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:07.250855 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:07.250856 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:07.250857 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 15:03:07.250896 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:07.250898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.251742 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.251743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:07.252330 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:07.252332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.252797 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.252798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:07.252869 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:07.252887 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 15:03:07.253050 (+ 163us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 15:03:07.253055 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 15:03:07.253099 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 15:03:07.253146 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 15:03:07.254091 2990 raft_consensus.cc:1261] T d7239eebe59243dd914382170ae34e60 P 10f59698d1e543faab4809d8ccd21971 [term 1 FOLLOWER]: Refusing update from remote peer f103800ea1744a2fa4e7e3ac01bee388: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:07.254172 3184 raft_consensus.cc:1261] T d7239eebe59243dd914382170ae34e60 P cf8789339ccc43b98ef3f890059006d3 [term 1 FOLLOWER]: Refusing update from remote peer f103800ea1744a2fa4e7e3ac01bee388: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:07.254276 3291 consensus_queue.cc:1035] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:07.254360 3293 consensus_queue.cc:1035] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:07.254779 3306 tablet_replica.cc:439] T d7239eebe59243dd914382170ae34e60 P 10f59698d1e543faab4809d8ccd21971: TxnStatusTablet state changed. Reason: New leader f103800ea1744a2fa4e7e3ac01bee388. Latest consensus state: current_term: 1 leader_uuid: "f103800ea1744a2fa4e7e3ac01bee388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } } }
I20260806 15:03:07.254855 3306 tablet_replica.cc:442] T d7239eebe59243dd914382170ae34e60 P 10f59698d1e543faab4809d8ccd21971: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:03:07.255131 3290 tablet_replica.cc:439] T d7239eebe59243dd914382170ae34e60 P cf8789339ccc43b98ef3f890059006d3: TxnStatusTablet state changed. Reason: New leader f103800ea1744a2fa4e7e3ac01bee388. Latest consensus state: current_term: 1 leader_uuid: "f103800ea1744a2fa4e7e3ac01bee388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } } }
I20260806 15:03:07.255214 3290 tablet_replica.cc:442] T d7239eebe59243dd914382170ae34e60 P cf8789339ccc43b98ef3f890059006d3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:03:07.255440 3296 mvcc.cc:205] Tried to move back new op lower bound from 7315573093391396864 to 7315573093313642496. Current Snapshot: MvccSnapshot[applied={T|T < 7315573093391396864}]
I20260806 15:03:07.255537 3293 tablet_replica.cc:439] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f103800ea1744a2fa4e7e3ac01bee388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } } }
I20260806 15:03:07.255594 3293 tablet_replica.cc:442] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388: This TxnStatusTablet replica's current role is: LEADER
I20260806 15:03:07.255617 3304 mvcc.cc:205] Tried to move back new op lower bound from 7315573093391396864 to 7315573093313642496. Current Snapshot: MvccSnapshot[applied={T|T < 7315573093391396864}]
I20260806 15:03:07.255769 3306 tablet_replica.cc:439] T d7239eebe59243dd914382170ae34e60 P 10f59698d1e543faab4809d8ccd21971: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f103800ea1744a2fa4e7e3ac01bee388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } } }
I20260806 15:03:07.255833 3306 tablet_replica.cc:442] T d7239eebe59243dd914382170ae34e60 P 10f59698d1e543faab4809d8ccd21971: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:03:07.256395 3299 tablet_replica.cc:439] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f103800ea1744a2fa4e7e3ac01bee388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } } }
I20260806 15:03:07.256492 3299 tablet_replica.cc:442] T d7239eebe59243dd914382170ae34e60 P f103800ea1744a2fa4e7e3ac01bee388: This TxnStatusTablet replica's current role is: LEADER
I20260806 15:03:07.256668 3290 tablet_replica.cc:439] T d7239eebe59243dd914382170ae34e60 P cf8789339ccc43b98ef3f890059006d3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f103800ea1744a2fa4e7e3ac01bee388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 } } peers { permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 } } peers { permanent_uuid: "cf8789339ccc43b98ef3f890059006d3" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37323 } } }
I20260806 15:03:07.256742 3290 tablet_replica.cc:442] T d7239eebe59243dd914382170ae34e60 P cf8789339ccc43b98ef3f890059006d3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:03:07.259997 3272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:07.257764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34020 (local address 127.0.236.67:37323)
0806 15:03:07.257803 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:07.257805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:07.257813 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:07.257846 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:07.257846 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:07.257864 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:07.257973 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:03:07.258075 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.258532 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.259242 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.259546 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.259609 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:07.259707 (+ 98us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:03:07.259832 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:03:07.259898 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:07.259913 (+ 15us) server_negotiation.cc:299] Negotiation successful
0806 15:03:07.259932 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 15:03:07.264670 3258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:07.262186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:39611 (local address 127.0.236.67:54195)
0806 15:03:07.262253 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:07.262264 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:07.262305 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:07.262519 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:07.262519 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:07.262520 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 15:03:07.262575 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:07.262577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.263102 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.263103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:07.263987 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:07.263990 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.264271 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.264272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:07.264334 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:07.264350 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 15:03:07.264555 (+ 205us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 15:03:07.264558 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:03:07.264591 (+ 33us) client_negotiation.cc:240] Negotiation successful
0806 15:03:07.264626 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 15:03:07.264680 3277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:07.262375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:54195 (local address 127.0.236.65:39611)
0806 15:03:07.262427 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:07.262429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:07.262435 (+ 6us) server_negotiation.cc:407] Connection header received
0806 15:03:07.262451 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:07.262452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:07.262459 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:07.262499 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:03:07.262632 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.263049 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.264054 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.264218 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.264257 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:07.264474 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:03:07.264507 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:03:07.264552 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:07.264596 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 15:03:07.264616 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 15:03:07.266901 2990 raft_consensus.cc:1261] T 8a3f703bbb0b407cac538b4c1e7e8357 P 10f59698d1e543faab4809d8ccd21971 [term 1 FOLLOWER]: Refusing update from remote peer cf8789339ccc43b98ef3f890059006d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:07.266911 2782 raft_consensus.cc:1261] T 8a3f703bbb0b407cac538b4c1e7e8357 P f103800ea1744a2fa4e7e3ac01bee388 [term 1 FOLLOWER]: Refusing update from remote peer cf8789339ccc43b98ef3f890059006d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:07.267153 3321 consensus_queue.cc:1035] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10f59698d1e543faab4809d8ccd21971" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:07.267271 3290 consensus_queue.cc:1035] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f103800ea1744a2fa4e7e3ac01bee388" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 39611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:07.268538 3297 mvcc.cc:205] Tried to move back new op lower bound from 7315573093443407872 to 7315573092811796480. Current Snapshot: MvccSnapshot[applied={T|T < 7315573093443407872}]
I20260806 15:03:07.275717 3359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:07.273253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:37323 (local address 127.0.236.65:36219)
0806 15:03:07.273412 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:07.273422 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:07.273470 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:07.273573 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:07.273573 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:07.273574 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 15:03:07.273614 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:07.273615 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.274232 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.274233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:07.274977 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:07.274980 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.275416 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.275417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:07.275456 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:07.275469 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 15:03:07.275601 (+ 132us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 15:03:07.275602 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 15:03:07.275630 (+ 28us) client_negotiation.cc:240] Negotiation successful
0806 15:03:07.275661 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260806 15:03:07.275713 3272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:07.273329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:36219 (local address 127.0.236.67:37323)
0806 15:03:07.273367 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:07.273369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:07.273473 (+ 104us) server_negotiation.cc:407] Connection header received
0806 15:03:07.273499 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:07.273500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:07.273508 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:07.273571 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:03:07.273671 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.274158 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.275065 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.275357 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.275418 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:07.275528 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:03:07.275557 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:03:07.275602 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:07.275634 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 15:03:07.275651 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 15:03:07.284858 2560 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:07.284945 2560 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:07.285540 2560 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7e616f8dd9b4ba19a577c0faa18bfa7: Sending DeleteTablet for 3 replicas of tablet 8a3f703bbb0b407cac538b4c1e7e8357
I20260806 15:03:07.285786 3158 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a3f703bbb0b407cac538b4c1e7e8357 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:07 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:34008
I20260806 15:03:07.285791 2959 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a3f703bbb0b407cac538b4c1e7e8357 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:07 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:56296
I20260806 15:03:07.285868 3311 tablet_replica.cc:330] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3: stopping tablet replica
I20260806 15:03:07.285872 3312 tablet_replica.cc:330] T 8a3f703bbb0b407cac538b4c1e7e8357 P 10f59698d1e543faab4809d8ccd21971: stopping tablet replica
I20260806 15:03:07.285933 3311 raft_consensus.cc:2229] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3 [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:03:07.285938 3312 raft_consensus.cc:2229] T 8a3f703bbb0b407cac538b4c1e7e8357 P 10f59698d1e543faab4809d8ccd21971 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:07.285970 3312 raft_consensus.cc:2258] T 8a3f703bbb0b407cac538b4c1e7e8357 P 10f59698d1e543faab4809d8ccd21971 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:07.286017 3311 raft_consensus.cc:2258] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:07.286288 3311 ts_tablet_manager.cc:1905] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:07.287113 3312 ts_tablet_manager.cc:1905] T 8a3f703bbb0b407cac538b4c1e7e8357 P 10f59698d1e543faab4809d8ccd21971: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:07.287802 3311 ts_tablet_manager.cc:1918] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:03:07.287851 3311 log.cc:1196] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/8a3f703bbb0b407cac538b4c1e7e8357
I20260806 15:03:07.288071 3311 ts_tablet_manager.cc:1939] T 8a3f703bbb0b407cac538b4c1e7e8357 P cf8789339ccc43b98ef3f890059006d3: Deleting consensus metadata
I20260806 15:03:07.288386 2536 catalog_manager.cc:4841] TS cf8789339ccc43b98ef3f890059006d3 (127.0.236.67:37323): tablet 8a3f703bbb0b407cac538b4c1e7e8357 (table test-table [id=ae17d168a1b94361b0038f4807add155]) successfully deleted
I20260806 15:03:07.288522 2754 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a3f703bbb0b407cac538b4c1e7e8357 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:07 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:51406
I20260806 15:03:07.288590 3313 tablet_replica.cc:330] T 8a3f703bbb0b407cac538b4c1e7e8357 P f103800ea1744a2fa4e7e3ac01bee388: stopping tablet replica
I20260806 15:03:07.288633 3313 raft_consensus.cc:2229] T 8a3f703bbb0b407cac538b4c1e7e8357 P f103800ea1744a2fa4e7e3ac01bee388 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:07.288642 3312 ts_tablet_manager.cc:1918] T 8a3f703bbb0b407cac538b4c1e7e8357 P 10f59698d1e543faab4809d8ccd21971: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:03:07.288661 3313 raft_consensus.cc:2258] T 8a3f703bbb0b407cac538b4c1e7e8357 P f103800ea1744a2fa4e7e3ac01bee388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:07.288693 3312 log.cc:1196] T 8a3f703bbb0b407cac538b4c1e7e8357 P 10f59698d1e543faab4809d8ccd21971: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-1/wal/wals/8a3f703bbb0b407cac538b4c1e7e8357
I20260806 15:03:07.288887 3312 ts_tablet_manager.cc:1939] T 8a3f703bbb0b407cac538b4c1e7e8357 P 10f59698d1e543faab4809d8ccd21971: Deleting consensus metadata
I20260806 15:03:07.288940 3313 ts_tablet_manager.cc:1905] T 8a3f703bbb0b407cac538b4c1e7e8357 P f103800ea1744a2fa4e7e3ac01bee388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:07.289176 2533 catalog_manager.cc:4841] TS 10f59698d1e543faab4809d8ccd21971 (127.0.236.66:42819): tablet 8a3f703bbb0b407cac538b4c1e7e8357 (table test-table [id=ae17d168a1b94361b0038f4807add155]) successfully deleted
I20260806 15:03:07.290359 3313 ts_tablet_manager.cc:1918] T 8a3f703bbb0b407cac538b4c1e7e8357 P f103800ea1744a2fa4e7e3ac01bee388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:03:07.290421 3313 log.cc:1196] T 8a3f703bbb0b407cac538b4c1e7e8357 P f103800ea1744a2fa4e7e3ac01bee388: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028580791223-945-0/minicluster-data/ts-0/wal/wals/8a3f703bbb0b407cac538b4c1e7e8357
I20260806 15:03:07.290637 3313 ts_tablet_manager.cc:1939] T 8a3f703bbb0b407cac538b4c1e7e8357 P f103800ea1744a2fa4e7e3ac01bee388: Deleting consensus metadata
I20260806 15:03:07.290926 2536 catalog_manager.cc:4841] TS f103800ea1744a2fa4e7e3ac01bee388 (127.0.236.65:39611): tablet 8a3f703bbb0b407cac538b4c1e7e8357 (table test-table [id=ae17d168a1b94361b0038f4807add155]) successfully deleted
Aug 06 15:03:07 dist-test-slave-tc2s krb5kdc[2490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028586, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.236.65@KRBTEST.COM
I20260806 15:03:07.299782 3277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:07.289637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51440 (local address 127.0.236.65:39611)
0806 15:03:07.289673 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:07.289675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:07.289909 (+ 234us) server_negotiation.cc:407] Connection header received
0806 15:03:07.290009 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:07.290010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:07.290019 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:07.290045 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:07.290918 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.291604 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.292682 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.292960 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.293029 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:07.296512 (+ 3483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:07.296521 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:07.296521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:07.296528 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:07.298523 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:07.298906 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:07.298910 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:07.298910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:07.298982 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:07.299176 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:07.299177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:07.299177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:07.299390 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:07.299538 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:07.299616 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 15:03:07.299695 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 15:03:07.300093 2831 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51440
I20260806 15:03:07.308187 2628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:07.301092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40868 (local address 127.0.236.126:39061)
0806 15:03:07.301141 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:07.301143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:07.301376 (+ 233us) server_negotiation.cc:407] Connection header received
0806 15:03:07.301419 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:07.301420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:07.301430 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:07.301490 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:07.302075 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.302788 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.303787 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:07.304059 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:07.304134 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:07.305405 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:07.305407 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:07.305407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:07.305415 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:07.307029 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:07.307411 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:07.307412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:07.307412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:07.307487 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:07.307677 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:07.307678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:07.307679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:07.307811 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:07.307908 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:07.308026 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 15:03:07.308106 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260806 15:03:07.308471 2560 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40868
I20260806 15:03:07.309079 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 2659
I20260806 15:03:07.318264 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 2848
I20260806 15:03:07.334064 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 3058
I20260806 15:03:07.341423 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 2508
2026-08-06T15:03:07Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2299 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:03:07 dist-test-slave-tc2s krb5kdc[3373](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:03:07 dist-test-slave-tc2s krb5kdc[3373](info): set up 2 sockets
Aug 06 15:03:07 dist-test-slave-tc2s krb5kdc[3373](info): commencing operation
krb5kdc: starting...
W20260806 15:03:08.452809 945 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:03:08 dist-test-slave-tc2s krb5kdc[3373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:03:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:03:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:03:08.560366 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:39493
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:46565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:39493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:08.639526 3393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:08.639712 3393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:08.639734 3393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:08.640803 3393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:03:08.640838 3393 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 15:03:08.640851 3393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:08.640861 3393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:08.640878 3393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:03:08.640894 3393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:03:08.642331 3393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:46565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:39493
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:39493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:08.642517 3393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:08.642678 3393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 15:03:08.647076 3393 server_base.cc:1034] running on GCE node
W20260806 15:03:08.647055 3399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:08.647112 3400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:08.647060 3403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:08.647535 3393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:08.647809 3393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:08.648957 3393 hybrid_clock.cc:648] HybridClock initialized: now 1786028588648937 us; error 32 us; skew 500 ppm
Aug 06 15:03:08 dist-test-slave-tc2s krb5kdc[3373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028588, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:08.651412 3393 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:03:08.651854 3393 webserver.cc:460] Webserver started at http://127.0.236.126:35421/ using document root <none> and password file <none>
I20260806 15:03:08.652058 3393 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:08.652101 3393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:08.652220 3393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:08.653157 3393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "0dba63e943394e3ab6a9a29709510920"
format_stamp: "Formatted at 2026-08-06 15:03:08 on dist-test-slave-tc2s"
server_key: "7c32a38a2b9dcb45f8afed0399b25c42"
server_key_iv: "a9f3c6b28fd00abd71e96ee5c7eea5e2"
server_key_version: "encryptionkey@0"
I20260806 15:03:08.653468 3393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "0dba63e943394e3ab6a9a29709510920"
format_stamp: "Formatted at 2026-08-06 15:03:08 on dist-test-slave-tc2s"
server_key: "7c32a38a2b9dcb45f8afed0399b25c42"
server_key_iv: "a9f3c6b28fd00abd71e96ee5c7eea5e2"
server_key_version: "encryptionkey@0"
I20260806 15:03:08.654703 3393 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:03:08.655316 3411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:08.655467 3393 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:08.655546 3393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "0dba63e943394e3ab6a9a29709510920"
format_stamp: "Formatted at 2026-08-06 15:03:08 on dist-test-slave-tc2s"
server_key: "7c32a38a2b9dcb45f8afed0399b25c42"
server_key_iv: "a9f3c6b28fd00abd71e96ee5c7eea5e2"
server_key_version: "encryptionkey@0"
I20260806 15:03:08.655599 3393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:08.663331 3393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:08.664914 3393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:08.665010 3393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:08.669224 3393 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:39493
I20260806 15:03:08.669297 3505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:39493 every 8 connection(s)
I20260806 15:03:08.669540 3393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:03:08.670167 3506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:08.672158 3506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920: Bootstrap starting.
I20260806 15:03:08.672681 3506 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:08.672927 3506 log.cc:824] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:08.673527 3506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920: No bootstrap required, opened a new log
I20260806 15:03:08.674252 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 3393
I20260806 15:03:08.674331 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:03:08.674466 945 external_mini_cluster.cc:1428] Setting key 561889a001b7e16fd285c729b3987668
I20260806 15:03:08.675447 3506 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dba63e943394e3ab6a9a29709510920" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39493 } }
I20260806 15:03:08.675602 3506 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:08.675693 3506 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dba63e943394e3ab6a9a29709510920, State: Initialized, Role: FOLLOWER
I20260806 15:03:08.675827 3506 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dba63e943394e3ab6a9a29709510920" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39493 } }
I20260806 15:03:08.675904 3506 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:03:08.675926 3506 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:03:08.675980 3506 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:08.676546 3506 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dba63e943394e3ab6a9a29709510920" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39493 } }
I20260806 15:03:08.676651 3506 leader_election.cc:302] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0dba63e943394e3ab6a9a29709510920; no voters:
I20260806 15:03:08.676806 3506 leader_election.cc:288] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:03:08.676851 3517 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:08.676994 3517 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920 [term 1 LEADER]: Becoming Leader. State: Replica: 0dba63e943394e3ab6a9a29709510920, State: Running, Role: LEADER
I20260806 15:03:08.677034 3506 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:03:08.677150 3517 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dba63e943394e3ab6a9a29709510920" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39493 } }
I20260806 15:03:08.677469 3517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dba63e943394e3ab6a9a29709510920. Latest consensus state: current_term: 1 leader_uuid: "0dba63e943394e3ab6a9a29709510920" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dba63e943394e3ab6a9a29709510920" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39493 } } }
I20260806 15:03:08.677531 3526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0dba63e943394e3ab6a9a29709510920" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dba63e943394e3ab6a9a29709510920" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39493 } } }
I20260806 15:03:08.677536 3517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:08.677609 3526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920 [sys.catalog]: This master's current role is: LEADER
W20260806 15:03:08.678803 3543 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:03:08.678903 3543 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:03:08.679021 3534 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:03:08.679206 3534 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:03:08.680586 3534 catalog_manager.cc:1347] Generated new cluster ID: 7b24fb9e019d46bba33fb35717dea16b
I20260806 15:03:08.680635 3534 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 15:03:08 dist-test-slave-tc2s krb5kdc[3373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:08.685638 3513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:08.674874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60314 (local address 127.0.236.126:39493)
0806 15:03:08.675281 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:08.675290 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:08.675322 (+ 32us) server_negotiation.cc:407] Connection header received
0806 15:03:08.675429 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:08.675447 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:08.675502 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:08.675644 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:08.676288 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:08.677672 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:08.678759 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:08.679274 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:08.679378 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:08.682255 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:08.682258 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:08.682258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:08.682268 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:08.684026 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:08.684405 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:08.684408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:08.684408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:08.684532 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:08.684725 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:08.684725 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:08.684726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:08.684986 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:08.685113 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:08.685233 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 15:03:08.685456 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":86,"threads_started":1}
I20260806 15:03:08.691548 3534 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:03:08.692214 3534 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:03:08.710137 3534 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920: Generated new TSK 0
I20260806 15:03:08.710300 3534 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.236.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
I20260806 15:03:08.735895 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:0
--local_ip_for_outbound_sockets=127.0.236.65
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:39493
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:46565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:08.813802 3552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:08.814013 3552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:08.814038 3552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:08.815088 3552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:08.815122 3552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:08.815141 3552 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 15:03:08.815155 3552 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 15:03:08.815171 3552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:03:08.816538 3552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:46565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.236.126:39493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:08.816712 3552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:08.816876 3552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:08.821334 3562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:08.821334 3568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:08.821347 3563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:08.821516 3552 server_base.cc:1034] running on GCE node
I20260806 15:03:08.821686 3552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:08.821914 3552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:08.823060 3552 hybrid_clock.cc:648] HybridClock initialized: now 1786028588823046 us; error 31 us; skew 500 ppm
Aug 06 15:03:08 dist-test-slave-tc2s krb5kdc[3373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028588, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:08.825393 3552 init.cc:377] Logged in from keytab as kudu/127.0.236.65@KRBTEST.COM (short username kudu)
I20260806 15:03:08.825786 3552 webserver.cc:460] Webserver started at http://127.0.236.65:46267/ using document root <none> and password file <none>
I20260806 15:03:08.826031 3552 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:08.826083 3552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:08.826205 3552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:08.827129 3552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-0/data/instance:
uuid: "a622511a14e14400ae2513c64e3fc5cd"
format_stamp: "Formatted at 2026-08-06 15:03:08 on dist-test-slave-tc2s"
server_key: "ffc8d4cd21012b5a49c4c944f5d26f13"
server_key_iv: "c8801a7c74ff6a77dcbffd91df69faa7"
server_key_version: "encryptionkey@0"
I20260806 15:03:08.827440 3552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-0/wal/instance:
uuid: "a622511a14e14400ae2513c64e3fc5cd"
format_stamp: "Formatted at 2026-08-06 15:03:08 on dist-test-slave-tc2s"
server_key: "ffc8d4cd21012b5a49c4c944f5d26f13"
server_key_iv: "c8801a7c74ff6a77dcbffd91df69faa7"
server_key_version: "encryptionkey@0"
I20260806 15:03:08.828598 3552 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:03:08.829205 3575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:08.829351 3552 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:08.829429 3552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "a622511a14e14400ae2513c64e3fc5cd"
format_stamp: "Formatted at 2026-08-06 15:03:08 on dist-test-slave-tc2s"
server_key: "ffc8d4cd21012b5a49c4c944f5d26f13"
server_key_iv: "c8801a7c74ff6a77dcbffd91df69faa7"
server_key_version: "encryptionkey@0"
I20260806 15:03:08.829487 3552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:08.842710 3552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:08.844303 3552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:08.844422 3552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:08.844882 3552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:08.844930 3552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:08.844957 3552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:08.844974 3552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 15:03:08 dist-test-slave-tc2s krb5kdc[3373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028588, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:08.851012 3552 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:38279
I20260806 15:03:08.851104 3738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:38279 every 8 connection(s)
I20260806 15:03:08.851370 3552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:03:08.853708 3513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:08.845230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:41189 (local address 127.0.236.126:39493)
0806 15:03:08.845296 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:08.845298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:08.845491 (+ 193us) server_negotiation.cc:407] Connection header received
0806 15:03:08.845880 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:08.845881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:08.845914 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:08.845989 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:08.846685 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:08.847429 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:08.848847 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:08.849168 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:08.849244 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:08.851304 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:08.851306 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:08.851306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:08.851316 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:08.852654 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:08.852994 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:08.852995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:08.852995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:08.853054 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:08.853236 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:08.853236 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:08.853236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:08.853332 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:08.853376 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:08.853578 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 15:03:08.853645 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 15:03:08.853852 3588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:08.845216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39493 (local address 127.0.236.65:41189)
0806 15:03:08.845370 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:08.845408 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:08.845801 (+ 393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:08.846038 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:08.846041 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:08.846331 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:08.846585 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:08.846591 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:08.847525 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:08.847526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:08.848756 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:08.848760 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:08.849254 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:08.849255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:08.849386 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:08.849824 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:08.849833 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:08.851208 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:08.852723 (+ 1515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:08.852729 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:08.852730 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:08.852930 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:08.853100 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:08.853100 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:08.853101 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:08.853192 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:08.853374 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:08.853377 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:08.853481 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 15:03:08.853557 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 15:03:08.853670 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260806 15:03:08.854336 3739 heartbeater.cc:347] Connected to a master server at 127.0.236.126:39493
I20260806 15:03:08.854418 3739 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:08.854598 3739 heartbeater.cc:511] Master 127.0.236.126:39493 requested a full tablet report, sending...
I20260806 15:03:08.855122 3434 ts_manager.cc:194] Registered new tserver with Master: a622511a14e14400ae2513c64e3fc5cd (127.0.236.65:38279)
I20260806 15:03:08.855784 3434 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.65@KRBTEST.COM'} at 127.0.236.65:41189
I20260806 15:03:08.860086 3750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:08.855289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39493 (local address 127.0.236.65:44017)
0806 15:03:08.855422 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:08.855432 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:08.855487 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:08.855604 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:08.855604 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:08.855764 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:08.855808 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:08.855810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:08.856613 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:08.856614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:08.857104 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:08.857106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:08.857386 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:08.857387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:08.857434 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:08.857733 (+ 299us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:08.857736 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:08.858039 (+ 303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:08.859365 (+ 1326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:08.859366 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:08.859367 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:08.859532 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:08.859663 (+ 131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:08.859664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:08.859664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:08.859706 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:08.859859 (+ 153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:08.859860 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:08.859912 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 15:03:08.859956 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 15:03:08.860026 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
I20260806 15:03:08.860069 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 3552
I20260806 15:03:08.860155 3513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:08.855346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:44017 (local address 127.0.236.126:39493)
0806 15:03:08.855382 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:08.855384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:08.855485 (+ 101us) server_negotiation.cc:407] Connection header received
0806 15:03:08.855539 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:08.855540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:08.855549 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:08.855599 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:08.855873 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:08.856529 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:08.857165 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:08.857340 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:08.857384 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:08.858108 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:08.858109 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:08.858109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:08.858115 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:08.859274 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:08.859583 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:08.859583 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:08.859583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:08.859628 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:08.859737 (+ 109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:08.859737 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:08.859738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:08.859820 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:08.859861 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:08.859994 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 15:03:08.860073 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 15:03:08.860162 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-0/wal/instance
I20260806 15:03:08.860316 945 external_mini_cluster.cc:1428] Setting key d5e2fee70b2b017063eee36edff84539
WARNING: no policy specified for kudu/127.0.236.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
I20260806 15:03:08.882079 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:0
--local_ip_for_outbound_sockets=127.0.236.66
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:39493
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:46565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:08.959486 3754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:08.959669 3754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:08.959702 3754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:08.960800 3754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:08.960840 3754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:08.960867 3754 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 15:03:08.960882 3754 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 15:03:08.960906 3754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:03:08.962342 3754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:46565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.236.126:39493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:08.962554 3754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:08.962749 3754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:08.967424 3765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:08.967434 3759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:08.967428 3760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:08.967653 3754 server_base.cc:1034] running on GCE node
I20260806 15:03:08.967809 3754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:08.968003 3754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:08.969126 3754 hybrid_clock.cc:648] HybridClock initialized: now 1786028588969118 us; error 27 us; skew 500 ppm
Aug 06 15:03:08 dist-test-slave-tc2s krb5kdc[3373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028588, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:08.971347 3754 init.cc:377] Logged in from keytab as kudu/127.0.236.66@KRBTEST.COM (short username kudu)
I20260806 15:03:08.971728 3754 webserver.cc:460] Webserver started at http://127.0.236.66:36453/ using document root <none> and password file <none>
I20260806 15:03:08.971905 3754 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:08.971948 3754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:08.972062 3754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:08.972987 3754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-1/data/instance:
uuid: "09573bd7d8714d91bcc69fcad505f93e"
format_stamp: "Formatted at 2026-08-06 15:03:08 on dist-test-slave-tc2s"
server_key: "6b746cc8acffd24e5321f28246fabb94"
server_key_iv: "059bbe6a0ada27c912edcf84e72d1ec2"
server_key_version: "encryptionkey@0"
I20260806 15:03:08.973282 3754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-1/wal/instance:
uuid: "09573bd7d8714d91bcc69fcad505f93e"
format_stamp: "Formatted at 2026-08-06 15:03:08 on dist-test-slave-tc2s"
server_key: "6b746cc8acffd24e5321f28246fabb94"
server_key_iv: "059bbe6a0ada27c912edcf84e72d1ec2"
server_key_version: "encryptionkey@0"
I20260806 15:03:08.974504 3754 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:03:08.975131 3777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:08.975275 3754 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:08.975358 3754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "09573bd7d8714d91bcc69fcad505f93e"
format_stamp: "Formatted at 2026-08-06 15:03:08 on dist-test-slave-tc2s"
server_key: "6b746cc8acffd24e5321f28246fabb94"
server_key_iv: "059bbe6a0ada27c912edcf84e72d1ec2"
server_key_version: "encryptionkey@0"
I20260806 15:03:08.975414 3754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:08.989024 3754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:08.990621 3754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:08.990724 3754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:08.991122 3754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:08.991163 3754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:08.991189 3754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:08.991205 3754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:08.997503 3754 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:39387
Aug 06 15:03:08 dist-test-slave-tc2s krb5kdc[3373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028588, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:08.997633 3941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:39387 every 8 connection(s)
I20260806 15:03:08.997864 3754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
I20260806 15:03:09.000380 3513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:08.991572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:55545 (local address 127.0.236.126:39493)
0806 15:03:08.991612 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:08.991614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:08.991847 (+ 233us) server_negotiation.cc:407] Connection header received
0806 15:03:08.992292 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:08.992293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:08.992307 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:08.992366 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:08.993194 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:08.993978 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:08.995401 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:08.995701 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:08.995763 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:08.998208 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:08.998209 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:08.998210 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:08.998217 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:08.999376 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:08.999722 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:08.999723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:08.999723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:08.999767 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:08.999928 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:08.999929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:08.999929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:09.000021 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:09.000065 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:09.000253 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 15:03:09.000308 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:03:09.000530 3787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:08.991571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39493 (local address 127.0.236.66:55545)
0806 15:03:08.991743 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:08.991775 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:08.992199 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:08.992465 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:08.992471 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:08.992804 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:08.993088 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:08.993095 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:08.994064 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:08.994065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:08.995300 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:08.995303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:08.995789 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:08.995791 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:08.995939 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:08.996385 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:08.996394 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:08.998101 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:08.999444 (+ 1343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:08.999450 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:08.999451 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:08.999661 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:08.999801 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:08.999802 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:08.999802 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:08.999884 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:09.000063 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:09.000066 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:09.000166 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 15:03:09.000237 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 15:03:09.000358 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":105,"threads_started":1}
I20260806 15:03:09.000975 3942 heartbeater.cc:347] Connected to a master server at 127.0.236.126:39493
I20260806 15:03:09.001056 3942 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:09.001211 3942 heartbeater.cc:511] Master 127.0.236.126:39493 requested a full tablet report, sending...
I20260806 15:03:09.001556 3434 ts_manager.cc:194] Registered new tserver with Master: 09573bd7d8714d91bcc69fcad505f93e (127.0.236.66:39387)
I20260806 15:03:09.001981 3434 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.66@KRBTEST.COM'} at 127.0.236.66:55545
I20260806 15:03:09.005976 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 3754
I20260806 15:03:09.006081 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-1/wal/instance
I20260806 15:03:09.006265 945 external_mini_cluster.cc:1428] Setting key 415e46e286d5f864790bd8a86cd091be
I20260806 15:03:09.006785 3513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:09.001936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:43935 (local address 127.0.236.126:39493)
0806 15:03:09.001979 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:09.001981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:09.002204 (+ 223us) server_negotiation.cc:407] Connection header received
0806 15:03:09.002261 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:09.002262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:09.002273 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:09.002321 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:09.002610 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.003087 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.003676 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.003841 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.003883 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:09.004568 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:09.004569 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:09.004569 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:09.004575 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:09.005639 (+ 1064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:09.005948 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:09.005949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:09.005949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:09.006019 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:09.006204 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:09.006205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:09.006205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:09.006396 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:09.006480 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:09.006631 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 15:03:09.006696 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:03:09.006798 3953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:09.001848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39493 (local address 127.0.236.66:43935)
0806 15:03:09.002135 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:09.002145 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:09.002205 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:09.002335 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:09.002336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:09.002493 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:09.002546 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:09.002548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.003156 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.003156 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:09.003624 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:09.003625 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.003887 (+ 262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.003887 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:09.003944 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:09.004240 (+ 296us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:09.004244 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:09.004505 (+ 261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:09.005699 (+ 1194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:09.005700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:09.005700 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:09.005852 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:09.006082 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:09.006083 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:09.006083 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:09.006149 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:09.006474 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:09.006475 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:09.006538 (+ 63us) client_negotiation.cc:769] Sending connection context
0806 15:03:09.006597 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 15:03:09.006696 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":95,"threads_started":1}
WARNING: no policy specified for kudu/127.0.236.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
I20260806 15:03:09.027815 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:0
--local_ip_for_outbound_sockets=127.0.236.67
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:39493
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:46565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:09.105983 3959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:09.106156 3959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:09.106179 3959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:09.107219 3959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:09.107250 3959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:09.107268 3959 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 15:03:09.107281 3959 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 15:03:09.107296 3959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:03:09.108625 3959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:46565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.236.126:39493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:09.108800 3959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:09.108970 3959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 15:03:09.113456 3959 server_base.cc:1034] running on GCE node
W20260806 15:03:09.113405 3974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:09.113391 3972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:09.113371 3971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:09.113765 3959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:09.113993 3959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:09.115128 3959 hybrid_clock.cc:648] HybridClock initialized: now 1786028589115116 us; error 27 us; skew 500 ppm
Aug 06 15:03:09 dist-test-slave-tc2s krb5kdc[3373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028589, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:09.117378 3959 init.cc:377] Logged in from keytab as kudu/127.0.236.67@KRBTEST.COM (short username kudu)
I20260806 15:03:09.117754 3959 webserver.cc:460] Webserver started at http://127.0.236.67:33233/ using document root <none> and password file <none>
I20260806 15:03:09.117969 3959 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:09.118016 3959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:09.118132 3959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:09.119048 3959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-2/data/instance:
uuid: "50556646c4e349f2a5a143e4db4c9176"
format_stamp: "Formatted at 2026-08-06 15:03:09 on dist-test-slave-tc2s"
server_key: "6a535fcb31aa8f2bf211dfff4c01a325"
server_key_iv: "5f680e9b1bac9e17990d5c8a827f2930"
server_key_version: "encryptionkey@0"
I20260806 15:03:09.119345 3959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-2/wal/instance:
uuid: "50556646c4e349f2a5a143e4db4c9176"
format_stamp: "Formatted at 2026-08-06 15:03:09 on dist-test-slave-tc2s"
server_key: "6a535fcb31aa8f2bf211dfff4c01a325"
server_key_iv: "5f680e9b1bac9e17990d5c8a827f2930"
server_key_version: "encryptionkey@0"
I20260806 15:03:09.120622 3959 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:03:09.121205 3982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:09.121352 3959 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:09.121441 3959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "50556646c4e349f2a5a143e4db4c9176"
format_stamp: "Formatted at 2026-08-06 15:03:09 on dist-test-slave-tc2s"
server_key: "6a535fcb31aa8f2bf211dfff4c01a325"
server_key_iv: "5f680e9b1bac9e17990d5c8a827f2930"
server_key_version: "encryptionkey@0"
I20260806 15:03:09.121517 3959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:09.144491 3959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:09.146438 3959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:09.146562 3959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:09.146987 3959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:09.147027 3959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:09.147056 3959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:09.147073 3959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 15:03:09 dist-test-slave-tc2s krb5kdc[3373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028589, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:09.153595 3959 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:32945
I20260806 15:03:09.153652 4153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:32945 every 8 connection(s)
I20260806 15:03:09.153951 3959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
I20260806 15:03:09.156175 3513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:09.147419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:60717 (local address 127.0.236.126:39493)
0806 15:03:09.147491 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:09.147493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:09.147682 (+ 189us) server_negotiation.cc:407] Connection header received
0806 15:03:09.148258 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:09.148259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:09.148269 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:09.148317 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:09.149105 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.149866 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.151335 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.151587 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.151642 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:09.153786 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:09.153787 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:09.153787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:09.153796 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:09.155120 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:09.155468 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:09.155469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:09.155469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:09.155524 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:09.155723 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:09.155723 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:09.155724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:09.155823 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:09.155870 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:09.156056 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 15:03:09.156113 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 15:03:09.156296 3996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:09.147366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39493 (local address 127.0.236.67:60717)
0806 15:03:09.147548 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:09.147595 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:09.148149 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:09.148392 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:09.148396 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:09.148725 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:09.149007 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:09.149013 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.149978 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.149979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:09.151239 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:09.151242 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.151663 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.151664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:09.151809 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:09.152222 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:09.152241 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:09.153701 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:09.155183 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:09.155189 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:09.155191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:09.155407 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:09.155579 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:09.155580 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:09.155581 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:09.155678 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:09.155867 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:09.155870 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:09.155974 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 15:03:09.156043 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 15:03:09.156150 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":85,"threads_started":1}
I20260806 15:03:09.156754 4156 heartbeater.cc:347] Connected to a master server at 127.0.236.126:39493
I20260806 15:03:09.156867 4156 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:09.157028 4156 heartbeater.cc:511] Master 127.0.236.126:39493 requested a full tablet report, sending...
I20260806 15:03:09.157362 3436 ts_manager.cc:194] Registered new tserver with Master: 50556646c4e349f2a5a143e4db4c9176 (127.0.236.67:32945)
I20260806 15:03:09.157799 3436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.67@KRBTEST.COM'} at 127.0.236.67:60717
I20260806 15:03:09.162091 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 3959
I20260806 15:03:09.162201 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-2/wal/instance
I20260806 15:03:09.162295 3513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:09.157733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:38767 (local address 127.0.236.126:39493)
0806 15:03:09.157772 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:09.157774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:09.157824 (+ 50us) server_negotiation.cc:407] Connection header received
0806 15:03:09.157882 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:09.157883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:09.157905 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:09.157963 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:09.158318 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.158806 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.159416 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.159583 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.159626 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:09.160275 (+ 649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:09.160276 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:09.160276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:09.160281 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:09.161456 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:09.161683 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:09.161684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:09.161684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:09.161724 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:09.161832 (+ 108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:09.161832 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:09.161832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:09.161948 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:09.161995 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:09.162143 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 15:03:09.162213 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:03:09.162313 4163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:09.157653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39493 (local address 127.0.236.67:38767)
0806 15:03:09.157769 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:09.157780 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:09.157836 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:09.157984 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:09.157985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:09.158170 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:09.158233 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:09.158235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.158886 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.158887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:09.159351 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:09.159352 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.159630 (+ 278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.159631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:09.159679 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:09.159977 (+ 298us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:09.159980 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:09.160214 (+ 234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:09.161510 (+ 1296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:09.161510 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:09.161511 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:09.161637 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:09.161757 (+ 120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:09.161757 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:09.161758 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:09.161801 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:09.162000 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:09.162000 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:09.162047 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 15:03:09.162125 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 15:03:09.162220 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":64,"threads_started":1}
I20260806 15:03:09.162390 945 external_mini_cluster.cc:1428] Setting key 407975e11b80a501d83bf5d5662b890f
I20260806 15:03:09.163255 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 15:03:09 dist-test-slave-tc2s krb5kdc[3373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 15:03:09 dist-test-slave-tc2s krb5kdc[3373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:09.184666 3513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:09.178996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60320 (local address 127.0.236.126:39493)
0806 15:03:09.179064 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:09.179065 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:09.179072 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:03:09.179131 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:09.179131 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:09.179139 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:09.179183 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:09.179658 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.180102 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.180754 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.180922 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.180961 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:09.182890 (+ 1929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:09.182891 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:09.182891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:09.182896 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:09.183877 (+ 981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:09.184169 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:09.184170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:09.184170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:09.184217 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:09.184338 (+ 121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:09.184339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:09.184339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:09.184422 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:09.184486 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:09.184559 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 15:03:09.184611 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 15:03:09.186028 3436 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60320:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:03:09.187408 3436 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:03:09.193105 4174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:09.189217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:39387 (local address 127.0.0.1:59658)
0806 15:03:09.189696 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:09.189705 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:09.189753 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:09.189949 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:09.189950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:09.189952 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:09.190005 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:09.190007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.190971 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.190972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:09.192068 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:09.192071 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.192822 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.192823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:09.192900 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:09.192912 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 15:03:09.192974 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 15:03:09.193033 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":2,"thread_start_us":48,"threads_started":1}
I20260806 15:03:09.193123 4179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:09.189521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59658 (local address 127.0.236.66:39387)
0806 15:03:09.189683 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:09.189686 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:09.189746 (+ 60us) server_negotiation.cc:407] Connection header received
0806 15:03:09.189825 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:09.189827 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:09.189841 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:09.189926 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:09.190144 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.190891 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.192159 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.192751 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.192808 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:09.192862 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:09.192992 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 15:03:09.193030 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":93,"threads_started":1}
I20260806 15:03:09.193262 4177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:09.189217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:32945 (local address 127.0.0.1:53618)
0806 15:03:09.189520 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:09.189559 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:09.189656 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:09.189850 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:09.189853 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:09.189855 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:09.189947 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:09.189949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.191187 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.191188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:09.192066 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:09.192069 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.192850 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.192851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:09.192924 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:09.192928 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:03:09.192981 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 15:03:09.193214 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":230,"thread_start_us":53,"threads_started":1}
I20260806 15:03:09.193392 4176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:09.189290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53618 (local address 127.0.236.67:32945)
0806 15:03:09.189457 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:09.189462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:09.189620 (+ 158us) server_negotiation.cc:407] Connection header received
0806 15:03:09.189743 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:09.189746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:09.189761 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:09.189838 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:09.190037 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.191093 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.192153 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.192698 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.192743 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:09.192793 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:09.193025 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 15:03:09.193059 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":83,"threads_started":1}
I20260806 15:03:09.193809 3847 tablet_service.cc:1467] Processing CreateTablet for tablet f57425826d314a64ba07b825ab6921e5 (DEFAULT_TABLE table=test-table [id=1347fbcbf62e4a6d983b43a17d6db81a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:09.193918 4061 tablet_service.cc:1467] Processing CreateTablet for tablet f57425826d314a64ba07b825ab6921e5 (DEFAULT_TABLE table=test-table [id=1347fbcbf62e4a6d983b43a17d6db81a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:09.194173 3847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f57425826d314a64ba07b825ab6921e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:09.194242 4175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:09.189389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:38279 (local address 127.0.0.1:50162)
0806 15:03:09.189812 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:09.189820 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:09.189856 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:09.190131 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:09.190131 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:09.190133 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:09.190168 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:09.190170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.191354 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.191355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:09.192406 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:09.192418 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.194017 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.194018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:09.194079 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:09.194084 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 15:03:09.194138 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 15:03:09.194179 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":60,"threads_started":1}
I20260806 15:03:09.194272 4061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f57425826d314a64ba07b825ab6921e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:09.194485 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:09.189488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50162 (local address 127.0.236.65:38279)
0806 15:03:09.189629 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:09.189634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:09.189927 (+ 293us) server_negotiation.cc:407] Connection header received
0806 15:03:09.189973 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:09.189976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:09.189992 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:09.190058 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:09.190241 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.191276 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.192497 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.193453 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.193514 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:09.193570 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:09.194348 (+ 778us) server_negotiation.cc:299] Negotiation successful
0806 15:03:09.194394 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
I20260806 15:03:09.196545 4180 tablet_bootstrap.cc:492] T f57425826d314a64ba07b825ab6921e5 P 50556646c4e349f2a5a143e4db4c9176: Bootstrap starting.
I20260806 15:03:09.196877 4182 tablet_bootstrap.cc:492] T f57425826d314a64ba07b825ab6921e5 P 09573bd7d8714d91bcc69fcad505f93e: Bootstrap starting.
I20260806 15:03:09.197417 4182 tablet_bootstrap.cc:654] T f57425826d314a64ba07b825ab6921e5 P 09573bd7d8714d91bcc69fcad505f93e: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:09.197465 4180 tablet_bootstrap.cc:654] T f57425826d314a64ba07b825ab6921e5 P 50556646c4e349f2a5a143e4db4c9176: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:09.197361 3644 tablet_service.cc:1467] Processing CreateTablet for tablet f57425826d314a64ba07b825ab6921e5 (DEFAULT_TABLE table=test-table [id=1347fbcbf62e4a6d983b43a17d6db81a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:09.197705 3644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f57425826d314a64ba07b825ab6921e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:09.197703 4182 log.cc:824] T f57425826d314a64ba07b825ab6921e5 P 09573bd7d8714d91bcc69fcad505f93e: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:09.197759 4180 log.cc:824] T f57425826d314a64ba07b825ab6921e5 P 50556646c4e349f2a5a143e4db4c9176: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:09.198403 4180 tablet_bootstrap.cc:492] T f57425826d314a64ba07b825ab6921e5 P 50556646c4e349f2a5a143e4db4c9176: No bootstrap required, opened a new log
I20260806 15:03:09.198437 4182 tablet_bootstrap.cc:492] T f57425826d314a64ba07b825ab6921e5 P 09573bd7d8714d91bcc69fcad505f93e: No bootstrap required, opened a new log
I20260806 15:03:09.198473 4180 ts_tablet_manager.cc:1397] T f57425826d314a64ba07b825ab6921e5 P 50556646c4e349f2a5a143e4db4c9176: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 15:03:09.198499 4182 ts_tablet_manager.cc:1397] T f57425826d314a64ba07b825ab6921e5 P 09573bd7d8714d91bcc69fcad505f93e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:03:09.199805 4180 raft_consensus.cc:348] T f57425826d314a64ba07b825ab6921e5 P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } }
I20260806 15:03:09.199928 4180 raft_consensus.cc:374] T f57425826d314a64ba07b825ab6921e5 P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:09.199961 4180 raft_consensus.cc:729] T f57425826d314a64ba07b825ab6921e5 P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50556646c4e349f2a5a143e4db4c9176, State: Initialized, Role: FOLLOWER
I20260806 15:03:09.200067 4180 consensus_queue.cc:260] T f57425826d314a64ba07b825ab6921e5 P 50556646c4e349f2a5a143e4db4c9176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } }
I20260806 15:03:09.200291 4180 ts_tablet_manager.cc:1428] T f57425826d314a64ba07b825ab6921e5 P 50556646c4e349f2a5a143e4db4c9176: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 15:03:09.200420 4156 heartbeater.cc:503] Master 127.0.236.126:39493 was elected leader, sending a full tablet report...
I20260806 15:03:09.200601 4182 raft_consensus.cc:348] T f57425826d314a64ba07b825ab6921e5 P 09573bd7d8714d91bcc69fcad505f93e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } }
I20260806 15:03:09.200773 4182 raft_consensus.cc:374] T f57425826d314a64ba07b825ab6921e5 P 09573bd7d8714d91bcc69fcad505f93e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:09.200811 4182 raft_consensus.cc:729] T f57425826d314a64ba07b825ab6921e5 P 09573bd7d8714d91bcc69fcad505f93e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09573bd7d8714d91bcc69fcad505f93e, State: Initialized, Role: FOLLOWER
I20260806 15:03:09.200917 4182 consensus_queue.cc:260] T f57425826d314a64ba07b825ab6921e5 P 09573bd7d8714d91bcc69fcad505f93e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } }
I20260806 15:03:09.201186 4182 ts_tablet_manager.cc:1428] T f57425826d314a64ba07b825ab6921e5 P 09573bd7d8714d91bcc69fcad505f93e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 15:03:09.201315 3942 heartbeater.cc:503] Master 127.0.236.126:39493 was elected leader, sending a full tablet report...
I20260806 15:03:09.201661 4189 tablet_bootstrap.cc:492] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd: Bootstrap starting.
I20260806 15:03:09.202284 4189 tablet_bootstrap.cc:654] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:09.202548 4189 log.cc:824] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:09.203096 4189 tablet_bootstrap.cc:492] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd: No bootstrap required, opened a new log
I20260806 15:03:09.203152 4189 ts_tablet_manager.cc:1397] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 15:03:09.204540 4189 raft_consensus.cc:348] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } }
I20260806 15:03:09.204654 4189 raft_consensus.cc:374] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:09.204684 4189 raft_consensus.cc:729] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a622511a14e14400ae2513c64e3fc5cd, State: Initialized, Role: FOLLOWER
I20260806 15:03:09.204782 4189 consensus_queue.cc:260] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } }
I20260806 15:03:09.204991 4189 ts_tablet_manager.cc:1428] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 15:03:09.205034 3739 heartbeater.cc:503] Master 127.0.236.126:39493 was elected leader, sending a full tablet report...
I20260806 15:03:09.208997 4196 raft_consensus.cc:482] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:03:09.209069 4196 raft_consensus.cc:504] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } }
I20260806 15:03:09.209312 4196 leader_election.cc:288] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50556646c4e349f2a5a143e4db4c9176 (127.0.236.67:32945), 09573bd7d8714d91bcc69fcad505f93e (127.0.236.66:39387)
I20260806 15:03:09.211915 3588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:09.209450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:32945 (local address 127.0.236.65:48607)
0806 15:03:09.209502 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:09.209513 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:09.209555 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:09.209791 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:09.209792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:09.209793 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:09.209841 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:09.209842 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.210397 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.210398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:09.211105 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:09.211106 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.211694 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.211695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:09.211765 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:09.211768 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:03:09.211816 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 15:03:09.211852 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":1}
I20260806 15:03:09.211918 4176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:09.209540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:48607 (local address 127.0.236.67:32945)
0806 15:03:09.209576 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:09.209577 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:09.209585 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:09.209604 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:09.209604 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:09.209611 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:09.209666 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:09.209912 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.210333 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.211181 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.211625 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.211690 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:09.211698 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:09.211827 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 15:03:09.211860 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:03:09.211983 4179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:09.209627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:58757 (local address 127.0.236.66:39387)
0806 15:03:09.209681 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:09.209683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:09.209691 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:09.209708 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:09.209709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:09.209716 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:09.209758 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:09.209863 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.210328 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.211225 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.211679 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.211740 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:09.211747 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:09.211891 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 15:03:09.211930 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 15:03:09.212023 4197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:09.209450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:39387 (local address 127.0.236.65:58757)
0806 15:03:09.209590 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:09.209599 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:09.209637 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:09.209762 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:09.209762 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:09.209763 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:09.209808 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:09.209809 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.210394 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.210395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:09.211149 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:09.211151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.211752 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.211753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:09.211825 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:09.211834 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 15:03:09.211880 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 15:03:09.211916 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":25,"thread_start_us":74,"threads_started":1}
I20260806 15:03:09.212265 4087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f57425826d314a64ba07b825ab6921e5" candidate_uuid: "a622511a14e14400ae2513c64e3fc5cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50556646c4e349f2a5a143e4db4c9176" is_pre_election: true
I20260806 15:03:09.212322 3877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f57425826d314a64ba07b825ab6921e5" candidate_uuid: "a622511a14e14400ae2513c64e3fc5cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09573bd7d8714d91bcc69fcad505f93e" is_pre_election: true
I20260806 15:03:09.212404 4087 raft_consensus.cc:2454] T f57425826d314a64ba07b825ab6921e5 P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a622511a14e14400ae2513c64e3fc5cd in term 0.
I20260806 15:03:09.212414 3877 raft_consensus.cc:2454] T f57425826d314a64ba07b825ab6921e5 P 09573bd7d8714d91bcc69fcad505f93e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a622511a14e14400ae2513c64e3fc5cd in term 0.
I20260806 15:03:09.212612 3577 leader_election.cc:302] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50556646c4e349f2a5a143e4db4c9176, a622511a14e14400ae2513c64e3fc5cd; no voters:
I20260806 15:03:09.212731 4196 raft_consensus.cc:2781] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:03:09.212793 4196 raft_consensus.cc:482] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:03:09.212810 4196 raft_consensus.cc:3037] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:09.213348 4196 raft_consensus.cc:504] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } }
I20260806 15:03:09.213454 4196 leader_election.cc:288] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd [CANDIDATE]: Term 1 election: Requested vote from peers 50556646c4e349f2a5a143e4db4c9176 (127.0.236.67:32945), 09573bd7d8714d91bcc69fcad505f93e (127.0.236.66:39387)
I20260806 15:03:09.213582 3877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f57425826d314a64ba07b825ab6921e5" candidate_uuid: "a622511a14e14400ae2513c64e3fc5cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09573bd7d8714d91bcc69fcad505f93e"
I20260806 15:03:09.213582 4087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f57425826d314a64ba07b825ab6921e5" candidate_uuid: "a622511a14e14400ae2513c64e3fc5cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50556646c4e349f2a5a143e4db4c9176"
I20260806 15:03:09.213650 4087 raft_consensus.cc:3037] T f57425826d314a64ba07b825ab6921e5 P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:09.213650 3877 raft_consensus.cc:3037] T f57425826d314a64ba07b825ab6921e5 P 09573bd7d8714d91bcc69fcad505f93e [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:09.214399 3877 raft_consensus.cc:2454] T f57425826d314a64ba07b825ab6921e5 P 09573bd7d8714d91bcc69fcad505f93e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622511a14e14400ae2513c64e3fc5cd in term 1.
I20260806 15:03:09.214399 4087 raft_consensus.cc:2454] T f57425826d314a64ba07b825ab6921e5 P 50556646c4e349f2a5a143e4db4c9176 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622511a14e14400ae2513c64e3fc5cd in term 1.
I20260806 15:03:09.214550 3577 leader_election.cc:302] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50556646c4e349f2a5a143e4db4c9176, a622511a14e14400ae2513c64e3fc5cd; no voters:
I20260806 15:03:09.214641 4196 raft_consensus.cc:2781] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:09.214745 4196 raft_consensus.cc:686] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd [term 1 LEADER]: Becoming Leader. State: Replica: a622511a14e14400ae2513c64e3fc5cd, State: Running, Role: LEADER
I20260806 15:03:09.214807 4196 consensus_queue.cc:237] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } }
I20260806 15:03:09.215452 3436 catalog_manager.cc:5495] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd reported cstate change: term changed from 0 to 1, leader changed from <none> to a622511a14e14400ae2513c64e3fc5cd (127.0.236.65). New cstate: current_term: 1 leader_uuid: "a622511a14e14400ae2513c64e3fc5cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } health_report { overall_health: HEALTHY } } }
Aug 06 15:03:09 dist-test-slave-tc2s krb5kdc[3373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028588, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:09.247982 4207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:09.242329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39493 (local address 127.0.0.1:60326)
0806 15:03:09.242467 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:09.242477 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:09.242534 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:09.242648 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:09.242649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:09.242919 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:09.242954 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:09.242957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.243545 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.243545 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:09.244032 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:09.244033 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.244319 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.244320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:09.244369 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:09.244763 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:09.244767 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:09.246021 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:09.247152 (+ 1131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:09.247158 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:09.247159 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:09.247340 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:09.247491 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:09.247491 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:09.247492 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:09.247541 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:09.247706 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:09.247711 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:09.247748 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 15:03:09.247793 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 15:03:09.247877 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":1,"thread_start_us":76,"threads_started":1}
I20260806 15:03:09.247984 3513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:09.242397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60326 (local address 127.0.236.126:39493)
0806 15:03:09.242454 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:09.242456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:09.242527 (+ 71us) server_negotiation.cc:407] Connection header received
0806 15:03:09.242585 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:09.242586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:09.242594 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:09.242640 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:09.243024 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.243478 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.244098 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.244265 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.244316 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:09.246105 (+ 1789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:09.246106 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:09.246107 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:09.246113 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:09.247091 (+ 978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:09.247396 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:09.247397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:09.247397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:09.247448 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:09.247582 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:09.247582 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:09.247583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:09.247667 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:09.247709 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:09.247809 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 15:03:09.247919 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":34}
W20260806 15:03:09.248371 3943 tablet.cc:2369] T f57425826d314a64ba07b825ab6921e5 P 09573bd7d8714d91bcc69fcad505f93e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:03:09.249629 3436 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:60326:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 15:03:09.249802 3436 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:03:09.251006 3644 tablet_service.cc:1467] Processing CreateTablet for tablet 40e2efaebe4c445b8ac1be264861b8ab (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c70e9040840c4a8db6687cf8a8d6496a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 15:03:09.251012 4061 tablet_service.cc:1467] Processing CreateTablet for tablet 40e2efaebe4c445b8ac1be264861b8ab (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c70e9040840c4a8db6687cf8a8d6496a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 15:03:09.251012 3847 tablet_service.cc:1467] Processing CreateTablet for tablet 40e2efaebe4c445b8ac1be264861b8ab (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c70e9040840c4a8db6687cf8a8d6496a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 15:03:09.251143 3847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40e2efaebe4c445b8ac1be264861b8ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:09.251143 4061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40e2efaebe4c445b8ac1be264861b8ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:09.251145 3644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40e2efaebe4c445b8ac1be264861b8ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:09.252460 4182 tablet_bootstrap.cc:492] T 40e2efaebe4c445b8ac1be264861b8ab P 09573bd7d8714d91bcc69fcad505f93e: Bootstrap starting.
I20260806 15:03:09.252516 4189 tablet_bootstrap.cc:492] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd: Bootstrap starting.
I20260806 15:03:09.252920 4189 tablet_bootstrap.cc:654] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:09.253146 4182 tablet_bootstrap.cc:654] T 40e2efaebe4c445b8ac1be264861b8ab P 09573bd7d8714d91bcc69fcad505f93e: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:09.253551 4180 tablet_bootstrap.cc:492] T 40e2efaebe4c445b8ac1be264861b8ab P 50556646c4e349f2a5a143e4db4c9176: Bootstrap starting.
I20260806 15:03:09.253623 4189 tablet_bootstrap.cc:492] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd: No bootstrap required, opened a new log
I20260806 15:03:09.253659 4189 ts_tablet_manager.cc:1397] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 15:03:09.253808 4182 tablet_bootstrap.cc:492] T 40e2efaebe4c445b8ac1be264861b8ab P 09573bd7d8714d91bcc69fcad505f93e: No bootstrap required, opened a new log
I20260806 15:03:09.253844 4182 ts_tablet_manager.cc:1397] T 40e2efaebe4c445b8ac1be264861b8ab P 09573bd7d8714d91bcc69fcad505f93e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:09.253788 4189 raft_consensus.cc:348] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:09.253871 4189 raft_consensus.cc:374] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:09.253911 4189 raft_consensus.cc:729] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a622511a14e14400ae2513c64e3fc5cd, State: Initialized, Role: FOLLOWER
I20260806 15:03:09.253950 4189 consensus_queue.cc:260] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:09.253990 4182 raft_consensus.cc:348] T 40e2efaebe4c445b8ac1be264861b8ab P 09573bd7d8714d91bcc69fcad505f93e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:09.254029 4189 ts_tablet_manager.cc:1428] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:09.254052 4182 raft_consensus.cc:374] T 40e2efaebe4c445b8ac1be264861b8ab P 09573bd7d8714d91bcc69fcad505f93e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:09.254069 4182 raft_consensus.cc:729] T 40e2efaebe4c445b8ac1be264861b8ab P 09573bd7d8714d91bcc69fcad505f93e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09573bd7d8714d91bcc69fcad505f93e, State: Initialized, Role: FOLLOWER
I20260806 15:03:09.254096 4180 tablet_bootstrap.cc:654] T 40e2efaebe4c445b8ac1be264861b8ab P 50556646c4e349f2a5a143e4db4c9176: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:09.254106 4182 consensus_queue.cc:260] T 40e2efaebe4c445b8ac1be264861b8ab P 09573bd7d8714d91bcc69fcad505f93e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:09.254122 4196 tablet_replica.cc:439] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } } }
I20260806 15:03:09.254179 4182 ts_tablet_manager.cc:1428] T 40e2efaebe4c445b8ac1be264861b8ab P 09573bd7d8714d91bcc69fcad505f93e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:09.254187 4196 tablet_replica.cc:442] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:03:09.254244 4188 tablet_replica.cc:439] T 40e2efaebe4c445b8ac1be264861b8ab P 09573bd7d8714d91bcc69fcad505f93e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } } }
I20260806 15:03:09.254303 4188 tablet_replica.cc:442] T 40e2efaebe4c445b8ac1be264861b8ab P 09573bd7d8714d91bcc69fcad505f93e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:03:09.254861 4180 tablet_bootstrap.cc:492] T 40e2efaebe4c445b8ac1be264861b8ab P 50556646c4e349f2a5a143e4db4c9176: No bootstrap required, opened a new log
I20260806 15:03:09.254912 4180 ts_tablet_manager.cc:1397] T 40e2efaebe4c445b8ac1be264861b8ab P 50556646c4e349f2a5a143e4db4c9176: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:09.255024 4180 raft_consensus.cc:348] T 40e2efaebe4c445b8ac1be264861b8ab P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:09.255076 4180 raft_consensus.cc:374] T 40e2efaebe4c445b8ac1be264861b8ab P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:09.255089 4180 raft_consensus.cc:729] T 40e2efaebe4c445b8ac1be264861b8ab P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50556646c4e349f2a5a143e4db4c9176, State: Initialized, Role: FOLLOWER
I20260806 15:03:09.255124 4180 consensus_queue.cc:260] T 40e2efaebe4c445b8ac1be264861b8ab P 50556646c4e349f2a5a143e4db4c9176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:09.255203 4180 ts_tablet_manager.cc:1428] T 40e2efaebe4c445b8ac1be264861b8ab P 50556646c4e349f2a5a143e4db4c9176: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:09.255275 4187 tablet_replica.cc:439] T 40e2efaebe4c445b8ac1be264861b8ab P 50556646c4e349f2a5a143e4db4c9176: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } } }
I20260806 15:03:09.255343 4187 tablet_replica.cc:442] T 40e2efaebe4c445b8ac1be264861b8ab P 50556646c4e349f2a5a143e4db4c9176: This TxnStatusTablet replica's current role is: FOLLOWER
W20260806 15:03:09.352109 3740 tablet.cc:2369] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:03:09.357709 4196 raft_consensus.cc:482] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:03:09.357765 4196 raft_consensus.cc:504] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:09.357954 4196 leader_election.cc:288] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09573bd7d8714d91bcc69fcad505f93e (127.0.236.66:39387), 50556646c4e349f2a5a143e4db4c9176 (127.0.236.67:32945)
I20260806 15:03:09.358120 4087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40e2efaebe4c445b8ac1be264861b8ab" candidate_uuid: "a622511a14e14400ae2513c64e3fc5cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50556646c4e349f2a5a143e4db4c9176" is_pre_election: true
I20260806 15:03:09.358119 3877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40e2efaebe4c445b8ac1be264861b8ab" candidate_uuid: "a622511a14e14400ae2513c64e3fc5cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09573bd7d8714d91bcc69fcad505f93e" is_pre_election: true
I20260806 15:03:09.358204 3877 raft_consensus.cc:2454] T 40e2efaebe4c445b8ac1be264861b8ab P 09573bd7d8714d91bcc69fcad505f93e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a622511a14e14400ae2513c64e3fc5cd in term 0.
I20260806 15:03:09.358206 4087 raft_consensus.cc:2454] T 40e2efaebe4c445b8ac1be264861b8ab P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a622511a14e14400ae2513c64e3fc5cd in term 0.
I20260806 15:03:09.358341 3581 leader_election.cc:302] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09573bd7d8714d91bcc69fcad505f93e, a622511a14e14400ae2513c64e3fc5cd; no voters:
I20260806 15:03:09.358448 4196 raft_consensus.cc:2781] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:03:09.358496 4196 raft_consensus.cc:482] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:03:09.358523 4196 raft_consensus.cc:3037] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:09.359124 4196 raft_consensus.cc:504] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:09.359239 4196 leader_election.cc:288] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd [CANDIDATE]: Term 1 election: Requested vote from peers 09573bd7d8714d91bcc69fcad505f93e (127.0.236.66:39387), 50556646c4e349f2a5a143e4db4c9176 (127.0.236.67:32945)
I20260806 15:03:09.359409 3877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40e2efaebe4c445b8ac1be264861b8ab" candidate_uuid: "a622511a14e14400ae2513c64e3fc5cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09573bd7d8714d91bcc69fcad505f93e"
I20260806 15:03:09.359422 4087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40e2efaebe4c445b8ac1be264861b8ab" candidate_uuid: "a622511a14e14400ae2513c64e3fc5cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50556646c4e349f2a5a143e4db4c9176"
I20260806 15:03:09.359485 3877 raft_consensus.cc:3037] T 40e2efaebe4c445b8ac1be264861b8ab P 09573bd7d8714d91bcc69fcad505f93e [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:09.359484 4087 raft_consensus.cc:3037] T 40e2efaebe4c445b8ac1be264861b8ab P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:09.360071 4087 raft_consensus.cc:2454] T 40e2efaebe4c445b8ac1be264861b8ab P 50556646c4e349f2a5a143e4db4c9176 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622511a14e14400ae2513c64e3fc5cd in term 1.
I20260806 15:03:09.360082 3877 raft_consensus.cc:2454] T 40e2efaebe4c445b8ac1be264861b8ab P 09573bd7d8714d91bcc69fcad505f93e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622511a14e14400ae2513c64e3fc5cd in term 1.
I20260806 15:03:09.360247 3577 leader_election.cc:302] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50556646c4e349f2a5a143e4db4c9176, a622511a14e14400ae2513c64e3fc5cd; no voters:
I20260806 15:03:09.360351 4196 raft_consensus.cc:2781] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:09.360410 4196 raft_consensus.cc:686] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd [term 1 LEADER]: Becoming Leader. State: Replica: a622511a14e14400ae2513c64e3fc5cd, State: Running, Role: LEADER
I20260806 15:03:09.360461 4196 consensus_queue.cc:237] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:09.360699 4198 tablet_replica.cc:439] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd: TxnStatusTablet state changed. Reason: New leader a622511a14e14400ae2513c64e3fc5cd. Latest consensus state: current_term: 1 leader_uuid: "a622511a14e14400ae2513c64e3fc5cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } } }
I20260806 15:03:09.360786 4198 tablet_replica.cc:442] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd: This TxnStatusTablet replica's current role is: LEADER
I20260806 15:03:09.360952 4214 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 15:03:09.361083 4214 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 15:03:09.360997 3436 catalog_manager.cc:5495] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd reported cstate change: term changed from 0 to 1, leader changed from <none> to a622511a14e14400ae2513c64e3fc5cd (127.0.236.65). New cstate: current_term: 1 leader_uuid: "a622511a14e14400ae2513c64e3fc5cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } health_report { overall_health: UNKNOWN } } }
I20260806 15:03:09.369288 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:09.366750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50172 (local address 127.0.236.65:38279)
0806 15:03:09.366789 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:09.366791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:09.366842 (+ 51us) server_negotiation.cc:407] Connection header received
0806 15:03:09.366948 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:09.366948 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:09.366963 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:09.367016 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:03:09.367132 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.367594 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.368640 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.368809 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.368849 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:09.369029 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:03:09.369113 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:03:09.369162 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:09.369223 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 15:03:09.369240 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:03:09.369342 4207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:09.366711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:38279 (local address 127.0.0.1:50172)
0806 15:03:09.366794 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:09.366806 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:09.366861 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:09.367024 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:09.367025 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:09.367026 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 15:03:09.367076 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:09.367078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.367668 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.367670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:09.368570 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:09.368573 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.368863 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.368864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:09.368934 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:09.368952 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 15:03:09.369163 (+ 211us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 15:03:09.369170 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 15:03:09.369220 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 15:03:09.369267 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 15:03:09.370792 3877 raft_consensus.cc:1261] T 40e2efaebe4c445b8ac1be264861b8ab P 09573bd7d8714d91bcc69fcad505f93e [term 1 FOLLOWER]: Refusing update from remote peer a622511a14e14400ae2513c64e3fc5cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:09.370797 4087 raft_consensus.cc:1261] T 40e2efaebe4c445b8ac1be264861b8ab P 50556646c4e349f2a5a143e4db4c9176 [term 1 FOLLOWER]: Refusing update from remote peer a622511a14e14400ae2513c64e3fc5cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:09.371088 4198 consensus_queue.cc:1035] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:09.371212 4196 consensus_queue.cc:1035] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:09.372388 4187 tablet_replica.cc:439] T 40e2efaebe4c445b8ac1be264861b8ab P 50556646c4e349f2a5a143e4db4c9176: TxnStatusTablet state changed. Reason: New leader a622511a14e14400ae2513c64e3fc5cd. Latest consensus state: current_term: 1 leader_uuid: "a622511a14e14400ae2513c64e3fc5cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } } }
I20260806 15:03:09.372388 4188 tablet_replica.cc:439] T 40e2efaebe4c445b8ac1be264861b8ab P 09573bd7d8714d91bcc69fcad505f93e: TxnStatusTablet state changed. Reason: New leader a622511a14e14400ae2513c64e3fc5cd. Latest consensus state: current_term: 1 leader_uuid: "a622511a14e14400ae2513c64e3fc5cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } } }
I20260806 15:03:09.372491 4187 tablet_replica.cc:442] T 40e2efaebe4c445b8ac1be264861b8ab P 50556646c4e349f2a5a143e4db4c9176: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:03:09.372491 4188 tablet_replica.cc:442] T 40e2efaebe4c445b8ac1be264861b8ab P 09573bd7d8714d91bcc69fcad505f93e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:03:09.373571 4198 tablet_replica.cc:439] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a622511a14e14400ae2513c64e3fc5cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } } }
I20260806 15:03:09.373648 4198 tablet_replica.cc:442] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd: This TxnStatusTablet replica's current role is: LEADER
I20260806 15:03:09.373921 4221 mvcc.cc:205] Tried to move back new op lower bound from 7315573102060818432 to 7315573102020829184. Current Snapshot: MvccSnapshot[applied={T|T < 7315573102060818432}]
I20260806 15:03:09.373997 4196 tablet_replica.cc:439] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a622511a14e14400ae2513c64e3fc5cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } } }
I20260806 15:03:09.374028 4220 mvcc.cc:205] Tried to move back new op lower bound from 7315573102060818432 to 7315573102020829184. Current Snapshot: MvccSnapshot[applied={T|T < 7315573102060818432}]
I20260806 15:03:09.374068 4196 tablet_replica.cc:442] T 40e2efaebe4c445b8ac1be264861b8ab P a622511a14e14400ae2513c64e3fc5cd: This TxnStatusTablet replica's current role is: LEADER
I20260806 15:03:09.374159 4187 tablet_replica.cc:439] T 40e2efaebe4c445b8ac1be264861b8ab P 50556646c4e349f2a5a143e4db4c9176: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a622511a14e14400ae2513c64e3fc5cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } } }
I20260806 15:03:09.374231 4187 tablet_replica.cc:442] T 40e2efaebe4c445b8ac1be264861b8ab P 50556646c4e349f2a5a143e4db4c9176: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:03:09.374462 4188 tablet_replica.cc:439] T 40e2efaebe4c445b8ac1be264861b8ab P 09573bd7d8714d91bcc69fcad505f93e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a622511a14e14400ae2513c64e3fc5cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } } }
I20260806 15:03:09.374540 4188 tablet_replica.cc:442] T 40e2efaebe4c445b8ac1be264861b8ab P 09573bd7d8714d91bcc69fcad505f93e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:03:09.374888 4222 mvcc.cc:205] Tried to move back new op lower bound from 7315573102060818432 to 7315573102020829184. Current Snapshot: MvccSnapshot[applied={T|T < 7315573102060818432}]
I20260806 15:03:09.378538 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:09.376044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50180 (local address 127.0.236.65:38279)
0806 15:03:09.376077 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:09.376079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:09.376102 (+ 23us) server_negotiation.cc:407] Connection header received
0806 15:03:09.376159 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:09.376159 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:09.376167 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:09.376209 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:03:09.376323 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.376739 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.377832 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.378143 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.378202 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:09.378353 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:03:09.378378 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:03:09.378413 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:09.378453 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 15:03:09.378473 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 15:03:09.383853 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:09.380888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:48031 (local address 127.0.236.65:38279)
0806 15:03:09.380919 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:09.380920 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:09.381080 (+ 160us) server_negotiation.cc:407] Connection header received
0806 15:03:09.381136 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:09.381137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:09.381143 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:09.381183 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:03:09.381305 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.381707 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.383298 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.383468 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.383511 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:09.383689 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:03:09.383709 (+ 20us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:03:09.383739 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:09.383802 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 15:03:09.383813 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 15:03:09.383958 4233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:09.380848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:38279 (local address 127.0.236.65:48031)
0806 15:03:09.381012 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:09.381023 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:09.381084 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:09.381188 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:09.381188 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:09.381190 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 15:03:09.381257 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:09.381258 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.381764 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.381765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:09.383214 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:09.383217 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:09.383527 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:09.383527 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:09.383594 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:09.383609 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 15:03:09.383753 (+ 144us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 15:03:09.383755 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 15:03:09.383801 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 15:03:09.383839 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
I20260806 15:03:09.386332 3877 raft_consensus.cc:1261] T f57425826d314a64ba07b825ab6921e5 P 09573bd7d8714d91bcc69fcad505f93e [term 1 FOLLOWER]: Refusing update from remote peer a622511a14e14400ae2513c64e3fc5cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:09.386365 4087 raft_consensus.cc:1261] T f57425826d314a64ba07b825ab6921e5 P 50556646c4e349f2a5a143e4db4c9176 [term 1 FOLLOWER]: Refusing update from remote peer a622511a14e14400ae2513c64e3fc5cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:09.386530 4235 consensus_queue.cc:1035] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:09.386612 4196 consensus_queue.cc:1035] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:09.400357 3436 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:09.400451 3436 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:09.401093 3436 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920: Sending DeleteTablet for 3 replicas of tablet f57425826d314a64ba07b825ab6921e5
I20260806 15:03:09.401407 4061 tablet_service.cc:1514] Processing DeleteTablet for tablet f57425826d314a64ba07b825ab6921e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:09 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:53618
I20260806 15:03:09.401407 3847 tablet_service.cc:1514] Processing DeleteTablet for tablet f57425826d314a64ba07b825ab6921e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:09 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:59658
I20260806 15:03:09.401605 3436 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60320:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 15:03:09.401684 4245 tablet_replica.cc:330] T f57425826d314a64ba07b825ab6921e5 P 09573bd7d8714d91bcc69fcad505f93e: stopping tablet replica
I20260806 15:03:09.401688 3643 tablet_service.cc:1514] Processing DeleteTablet for tablet f57425826d314a64ba07b825ab6921e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:09 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:50162
W20260806 15:03:09.401746 3436 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:03:09.401778 4245 raft_consensus.cc:2229] T f57425826d314a64ba07b825ab6921e5 P 09573bd7d8714d91bcc69fcad505f93e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:09.401827 4245 raft_consensus.cc:2258] T f57425826d314a64ba07b825ab6921e5 P 09573bd7d8714d91bcc69fcad505f93e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:09.402216 4246 tablet_replica.cc:330] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd: stopping tablet replica
I20260806 15:03:09.402262 4245 ts_tablet_manager.cc:1905] T f57425826d314a64ba07b825ab6921e5 P 09573bd7d8714d91bcc69fcad505f93e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:09.402397 4246 raft_consensus.cc:2229] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:03:09.402549 4246 raft_consensus.cc:2258] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:09.402878 3643 tablet_service.cc:1467] Processing CreateTablet for tablet 6c461f44093148739ac4a593a6f09d2c (DEFAULT_TABLE table=test-table [id=fad23af687fc4bedacbb081e5c7581b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:09.403000 3643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c461f44093148739ac4a593a6f09d2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:09.403132 4246 ts_tablet_manager.cc:1905] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:09.403824 4245 ts_tablet_manager.cc:1918] T f57425826d314a64ba07b825ab6921e5 P 09573bd7d8714d91bcc69fcad505f93e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:03:09.403904 4245 log.cc:1196] T f57425826d314a64ba07b825ab6921e5 P 09573bd7d8714d91bcc69fcad505f93e: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-1/wal/wals/f57425826d314a64ba07b825ab6921e5
I20260806 15:03:09.404138 4245 ts_tablet_manager.cc:1939] T f57425826d314a64ba07b825ab6921e5 P 09573bd7d8714d91bcc69fcad505f93e: Deleting consensus metadata
I20260806 15:03:09.404515 3417 catalog_manager.cc:4841] TS 09573bd7d8714d91bcc69fcad505f93e (127.0.236.66:39387): tablet f57425826d314a64ba07b825ab6921e5 (table test-table [id=1347fbcbf62e4a6d983b43a17d6db81a]) successfully deleted
I20260806 15:03:09.405006 4246 ts_tablet_manager.cc:1918] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:03:09.405141 4246 log.cc:1196] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-0/wal/wals/f57425826d314a64ba07b825ab6921e5
I20260806 15:03:09.405144 3847 tablet_service.cc:1467] Processing CreateTablet for tablet 6c461f44093148739ac4a593a6f09d2c (DEFAULT_TABLE table=test-table [id=fad23af687fc4bedacbb081e5c7581b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:09.405303 3847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c461f44093148739ac4a593a6f09d2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:09.405541 4246 ts_tablet_manager.cc:1939] T f57425826d314a64ba07b825ab6921e5 P a622511a14e14400ae2513c64e3fc5cd: Deleting consensus metadata
I20260806 15:03:09.406039 4061 tablet_service.cc:1467] Processing CreateTablet for tablet 6c461f44093148739ac4a593a6f09d2c (DEFAULT_TABLE table=test-table [id=fad23af687fc4bedacbb081e5c7581b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:09.406150 4061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c461f44093148739ac4a593a6f09d2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:09.407410 4180 tablet_bootstrap.cc:492] T 6c461f44093148739ac4a593a6f09d2c P 50556646c4e349f2a5a143e4db4c9176: Bootstrap starting.
I20260806 15:03:09.407866 4180 tablet_bootstrap.cc:654] T 6c461f44093148739ac4a593a6f09d2c P 50556646c4e349f2a5a143e4db4c9176: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:09.408614 4180 tablet_bootstrap.cc:492] T 6c461f44093148739ac4a593a6f09d2c P 50556646c4e349f2a5a143e4db4c9176: No bootstrap required, opened a new log
I20260806 15:03:09.408958 4182 tablet_bootstrap.cc:492] T 6c461f44093148739ac4a593a6f09d2c P 09573bd7d8714d91bcc69fcad505f93e: Bootstrap starting.
I20260806 15:03:09.408663 4180 ts_tablet_manager.cc:1397] T 6c461f44093148739ac4a593a6f09d2c P 50556646c4e349f2a5a143e4db4c9176: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:09.409332 4182 tablet_bootstrap.cc:654] T 6c461f44093148739ac4a593a6f09d2c P 09573bd7d8714d91bcc69fcad505f93e: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:09.409535 4180 raft_consensus.cc:348] T 6c461f44093148739ac4a593a6f09d2c P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:09.409658 4189 tablet_bootstrap.cc:492] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd: Bootstrap starting.
I20260806 15:03:09.409686 4242 tablet_replica.cc:330] T f57425826d314a64ba07b825ab6921e5 P 50556646c4e349f2a5a143e4db4c9176: stopping tablet replica
I20260806 15:03:09.409610 4180 raft_consensus.cc:374] T 6c461f44093148739ac4a593a6f09d2c P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:09.409761 4180 raft_consensus.cc:729] T 6c461f44093148739ac4a593a6f09d2c P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50556646c4e349f2a5a143e4db4c9176, State: Initialized, Role: FOLLOWER
I20260806 15:03:09.409819 4180 consensus_queue.cc:260] T 6c461f44093148739ac4a593a6f09d2c P 50556646c4e349f2a5a143e4db4c9176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:09.410034 4189 tablet_bootstrap.cc:654] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:09.410063 4180 ts_tablet_manager.cc:1428] T 6c461f44093148739ac4a593a6f09d2c P 50556646c4e349f2a5a143e4db4c9176: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 15:03:09.410898 4189 tablet_bootstrap.cc:492] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd: No bootstrap required, opened a new log
I20260806 15:03:09.410950 4189 ts_tablet_manager.cc:1397] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:09.411126 4189 raft_consensus.cc:348] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:09.411178 4189 raft_consensus.cc:374] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:09.411195 4189 raft_consensus.cc:729] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a622511a14e14400ae2513c64e3fc5cd, State: Initialized, Role: FOLLOWER
I20260806 15:03:09.411232 4189 consensus_queue.cc:260] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:09.411314 4189 ts_tablet_manager.cc:1428] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:09.409771 4242 raft_consensus.cc:2229] T f57425826d314a64ba07b825ab6921e5 P 50556646c4e349f2a5a143e4db4c9176 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:09.413972 3417 catalog_manager.cc:4841] TS a622511a14e14400ae2513c64e3fc5cd (127.0.236.65:38279): tablet f57425826d314a64ba07b825ab6921e5 (table test-table [id=1347fbcbf62e4a6d983b43a17d6db81a]) successfully deleted
I20260806 15:03:09.414022 4242 raft_consensus.cc:2258] T f57425826d314a64ba07b825ab6921e5 P 50556646c4e349f2a5a143e4db4c9176 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:09.414391 4182 tablet_bootstrap.cc:492] T 6c461f44093148739ac4a593a6f09d2c P 09573bd7d8714d91bcc69fcad505f93e: No bootstrap required, opened a new log
I20260806 15:03:09.414434 4182 ts_tablet_manager.cc:1397] T 6c461f44093148739ac4a593a6f09d2c P 09573bd7d8714d91bcc69fcad505f93e: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260806 15:03:09.414565 4182 raft_consensus.cc:348] T 6c461f44093148739ac4a593a6f09d2c P 09573bd7d8714d91bcc69fcad505f93e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:09.414630 4182 raft_consensus.cc:374] T 6c461f44093148739ac4a593a6f09d2c P 09573bd7d8714d91bcc69fcad505f93e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:09.414649 4182 raft_consensus.cc:729] T 6c461f44093148739ac4a593a6f09d2c P 09573bd7d8714d91bcc69fcad505f93e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09573bd7d8714d91bcc69fcad505f93e, State: Initialized, Role: FOLLOWER
I20260806 15:03:09.414690 4182 consensus_queue.cc:260] T 6c461f44093148739ac4a593a6f09d2c P 09573bd7d8714d91bcc69fcad505f93e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:09.414769 4182 ts_tablet_manager.cc:1428] T 6c461f44093148739ac4a593a6f09d2c P 09573bd7d8714d91bcc69fcad505f93e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:09.417548 4242 ts_tablet_manager.cc:1905] T f57425826d314a64ba07b825ab6921e5 P 50556646c4e349f2a5a143e4db4c9176: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:09.418903 4242 ts_tablet_manager.cc:1918] T f57425826d314a64ba07b825ab6921e5 P 50556646c4e349f2a5a143e4db4c9176: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:03:09.418973 4242 log.cc:1196] T f57425826d314a64ba07b825ab6921e5 P 50556646c4e349f2a5a143e4db4c9176: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/f57425826d314a64ba07b825ab6921e5
I20260806 15:03:09.419174 4242 ts_tablet_manager.cc:1939] T f57425826d314a64ba07b825ab6921e5 P 50556646c4e349f2a5a143e4db4c9176: Deleting consensus metadata
I20260806 15:03:09.419545 3415 catalog_manager.cc:4841] TS 50556646c4e349f2a5a143e4db4c9176 (127.0.236.67:32945): tablet f57425826d314a64ba07b825ab6921e5 (table test-table [id=1347fbcbf62e4a6d983b43a17d6db81a]) successfully deleted
I20260806 15:03:09.530831 4196 raft_consensus.cc:482] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:03:09.530916 4196 raft_consensus.cc:504] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:09.531075 4196 leader_election.cc:288] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09573bd7d8714d91bcc69fcad505f93e (127.0.236.66:39387), 50556646c4e349f2a5a143e4db4c9176 (127.0.236.67:32945)
I20260806 15:03:09.531253 3877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c461f44093148739ac4a593a6f09d2c" candidate_uuid: "a622511a14e14400ae2513c64e3fc5cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09573bd7d8714d91bcc69fcad505f93e" is_pre_election: true
I20260806 15:03:09.531358 3877 raft_consensus.cc:2454] T 6c461f44093148739ac4a593a6f09d2c P 09573bd7d8714d91bcc69fcad505f93e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a622511a14e14400ae2513c64e3fc5cd in term 0.
I20260806 15:03:09.531324 4087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c461f44093148739ac4a593a6f09d2c" candidate_uuid: "a622511a14e14400ae2513c64e3fc5cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50556646c4e349f2a5a143e4db4c9176" is_pre_election: true
I20260806 15:03:09.531402 4087 raft_consensus.cc:2454] T 6c461f44093148739ac4a593a6f09d2c P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a622511a14e14400ae2513c64e3fc5cd in term 0.
I20260806 15:03:09.531533 3581 leader_election.cc:302] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09573bd7d8714d91bcc69fcad505f93e, a622511a14e14400ae2513c64e3fc5cd; no voters:
I20260806 15:03:09.531618 4196 raft_consensus.cc:2781] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:03:09.531656 4196 raft_consensus.cc:482] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:03:09.531685 4196 raft_consensus.cc:3037] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:09.532351 4196 raft_consensus.cc:504] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:09.532474 4196 leader_election.cc:288] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd [CANDIDATE]: Term 1 election: Requested vote from peers 09573bd7d8714d91bcc69fcad505f93e (127.0.236.66:39387), 50556646c4e349f2a5a143e4db4c9176 (127.0.236.67:32945)
I20260806 15:03:09.532624 3877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c461f44093148739ac4a593a6f09d2c" candidate_uuid: "a622511a14e14400ae2513c64e3fc5cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09573bd7d8714d91bcc69fcad505f93e"
I20260806 15:03:09.532655 4087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c461f44093148739ac4a593a6f09d2c" candidate_uuid: "a622511a14e14400ae2513c64e3fc5cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50556646c4e349f2a5a143e4db4c9176"
I20260806 15:03:09.532699 3877 raft_consensus.cc:3037] T 6c461f44093148739ac4a593a6f09d2c P 09573bd7d8714d91bcc69fcad505f93e [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:09.532723 4087 raft_consensus.cc:3037] T 6c461f44093148739ac4a593a6f09d2c P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:09.533412 3877 raft_consensus.cc:2454] T 6c461f44093148739ac4a593a6f09d2c P 09573bd7d8714d91bcc69fcad505f93e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622511a14e14400ae2513c64e3fc5cd in term 1.
I20260806 15:03:09.533412 4087 raft_consensus.cc:2454] T 6c461f44093148739ac4a593a6f09d2c P 50556646c4e349f2a5a143e4db4c9176 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622511a14e14400ae2513c64e3fc5cd in term 1.
I20260806 15:03:09.533595 3577 leader_election.cc:302] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50556646c4e349f2a5a143e4db4c9176, a622511a14e14400ae2513c64e3fc5cd; no voters:
I20260806 15:03:09.533692 4196 raft_consensus.cc:2781] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:09.533738 4196 raft_consensus.cc:686] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd [term 1 LEADER]: Becoming Leader. State: Replica: a622511a14e14400ae2513c64e3fc5cd, State: Running, Role: LEADER
I20260806 15:03:09.533785 4196 consensus_queue.cc:237] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:09.534374 3436 catalog_manager.cc:5495] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd reported cstate change: term changed from 0 to 1, leader changed from <none> to a622511a14e14400ae2513c64e3fc5cd (127.0.236.65). New cstate: current_term: 1 leader_uuid: "a622511a14e14400ae2513c64e3fc5cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } health_report { overall_health: UNKNOWN } } }
I20260806 15:03:09.558523 4087 raft_consensus.cc:1261] T 6c461f44093148739ac4a593a6f09d2c P 50556646c4e349f2a5a143e4db4c9176 [term 1 FOLLOWER]: Refusing update from remote peer a622511a14e14400ae2513c64e3fc5cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:09.558535 3877 raft_consensus.cc:1261] T 6c461f44093148739ac4a593a6f09d2c P 09573bd7d8714d91bcc69fcad505f93e [term 1 FOLLOWER]: Refusing update from remote peer a622511a14e14400ae2513c64e3fc5cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:09.558713 4235 consensus_queue.cc:1035] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:09.558847 4196 consensus_queue.cc:1035] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 15:03:09.563628 4220 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 15:03:11.080490 3434 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:11.080593 3434 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:11.081362 3434 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920: Sending DeleteTablet for 3 replicas of tablet 6c461f44093148739ac4a593a6f09d2c
I20260806 15:03:11.081627 3847 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c461f44093148739ac4a593a6f09d2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:11 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:59658
I20260806 15:03:11.081646 3643 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c461f44093148739ac4a593a6f09d2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:11 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:50162
I20260806 15:03:11.081660 4061 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c461f44093148739ac4a593a6f09d2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:11 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:53618
I20260806 15:03:11.081854 4281 tablet_replica.cc:330] T 6c461f44093148739ac4a593a6f09d2c P 09573bd7d8714d91bcc69fcad505f93e: stopping tablet replica
I20260806 15:03:11.081946 4281 raft_consensus.cc:2229] T 6c461f44093148739ac4a593a6f09d2c P 09573bd7d8714d91bcc69fcad505f93e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:11.082376 4282 tablet_replica.cc:330] T 6c461f44093148739ac4a593a6f09d2c P 50556646c4e349f2a5a143e4db4c9176: stopping tablet replica
I20260806 15:03:11.082515 4283 tablet_replica.cc:330] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd: stopping tablet replica
I20260806 15:03:11.082592 4283 raft_consensus.cc:2229] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:03:11.082670 4282 raft_consensus.cc:2229] T 6c461f44093148739ac4a593a6f09d2c P 50556646c4e349f2a5a143e4db4c9176 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:11.082752 4283 raft_consensus.cc:2258] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:11.082813 4282 raft_consensus.cc:2258] T 6c461f44093148739ac4a593a6f09d2c P 50556646c4e349f2a5a143e4db4c9176 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:11.083113 4283 ts_tablet_manager.cc:1905] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:11.083278 4282 ts_tablet_manager.cc:1905] T 6c461f44093148739ac4a593a6f09d2c P 50556646c4e349f2a5a143e4db4c9176: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:11.083341 4281 raft_consensus.cc:2258] T 6c461f44093148739ac4a593a6f09d2c P 09573bd7d8714d91bcc69fcad505f93e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:11.084292 4281 ts_tablet_manager.cc:1905] T 6c461f44093148739ac4a593a6f09d2c P 09573bd7d8714d91bcc69fcad505f93e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:11.084942 4282 ts_tablet_manager.cc:1918] T 6c461f44093148739ac4a593a6f09d2c P 50556646c4e349f2a5a143e4db4c9176: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 15:03:11.085011 4282 log.cc:1196] T 6c461f44093148739ac4a593a6f09d2c P 50556646c4e349f2a5a143e4db4c9176: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/6c461f44093148739ac4a593a6f09d2c
I20260806 15:03:11.085234 4282 ts_tablet_manager.cc:1939] T 6c461f44093148739ac4a593a6f09d2c P 50556646c4e349f2a5a143e4db4c9176: Deleting consensus metadata
I20260806 15:03:11.085534 3415 catalog_manager.cc:4841] TS 50556646c4e349f2a5a143e4db4c9176 (127.0.236.67:32945): tablet 6c461f44093148739ac4a593a6f09d2c (table test-table [id=fad23af687fc4bedacbb081e5c7581b3]) successfully deleted
I20260806 15:03:11.085845 4281 ts_tablet_manager.cc:1918] T 6c461f44093148739ac4a593a6f09d2c P 09573bd7d8714d91bcc69fcad505f93e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 15:03:11.085927 4281 log.cc:1196] T 6c461f44093148739ac4a593a6f09d2c P 09573bd7d8714d91bcc69fcad505f93e: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-1/wal/wals/6c461f44093148739ac4a593a6f09d2c
I20260806 15:03:11.086170 4281 ts_tablet_manager.cc:1939] T 6c461f44093148739ac4a593a6f09d2c P 09573bd7d8714d91bcc69fcad505f93e: Deleting consensus metadata
I20260806 15:03:11.086465 3417 catalog_manager.cc:4841] TS 09573bd7d8714d91bcc69fcad505f93e (127.0.236.66:39387): tablet 6c461f44093148739ac4a593a6f09d2c (table test-table [id=fad23af687fc4bedacbb081e5c7581b3]) successfully deleted
I20260806 15:03:11.086908 4283 ts_tablet_manager.cc:1918] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 15:03:11.086962 4283 log.cc:1196] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-0/wal/wals/6c461f44093148739ac4a593a6f09d2c
I20260806 15:03:11.087146 4283 ts_tablet_manager.cc:1939] T 6c461f44093148739ac4a593a6f09d2c P a622511a14e14400ae2513c64e3fc5cd: Deleting consensus metadata
I20260806 15:03:11.087394 3417 catalog_manager.cc:4841] TS a622511a14e14400ae2513c64e3fc5cd (127.0.236.65:38279): tablet 6c461f44093148739ac4a593a6f09d2c (table test-table [id=fad23af687fc4bedacbb081e5c7581b3]) successfully deleted
Aug 06 15:03:11 dist-test-slave-tc2s krb5kdc[3373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028591, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 15:03:11 dist-test-slave-tc2s krb5kdc[3373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028591, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:11.107084 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:11.100433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60340 (local address 127.0.236.126:39493)
0806 15:03:11.100595 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:11.100598 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:11.100606 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:11.100663 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:11.100664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:11.100675 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:11.100730 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:11.101255 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:11.101855 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:11.102598 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:11.102800 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:11.102853 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:11.104880 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:11.104881 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:11.104881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:11.104887 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:11.106104 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:11.106393 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:11.106394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:11.106395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:11.106455 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:11.106617 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:11.106617 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:11.106618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:11.106724 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:11.106811 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:11.106913 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 15:03:11.106994 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":90,"threads_started":1}
I20260806 15:03:11.108202 3434 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60340:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:03:11.108368 3434 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:03:11.109493 3643 tablet_service.cc:1467] Processing CreateTablet for tablet f53f757c60044eb5a088864867b428fc (DEFAULT_TABLE table=test-table [id=aeb3e8a2e0ad49138d03d4a88fc75f14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:11.109539 3847 tablet_service.cc:1467] Processing CreateTablet for tablet f53f757c60044eb5a088864867b428fc (DEFAULT_TABLE table=test-table [id=aeb3e8a2e0ad49138d03d4a88fc75f14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:11.109613 3643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f53f757c60044eb5a088864867b428fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:11.109653 3847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f53f757c60044eb5a088864867b428fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:11.109993 4061 tablet_service.cc:1467] Processing CreateTablet for tablet f53f757c60044eb5a088864867b428fc (DEFAULT_TABLE table=test-table [id=aeb3e8a2e0ad49138d03d4a88fc75f14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:11.110105 4061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f53f757c60044eb5a088864867b428fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:11.110692 4297 tablet_bootstrap.cc:492] T f53f757c60044eb5a088864867b428fc P 09573bd7d8714d91bcc69fcad505f93e: Bootstrap starting.
I20260806 15:03:11.110941 4298 tablet_bootstrap.cc:492] T f53f757c60044eb5a088864867b428fc P a622511a14e14400ae2513c64e3fc5cd: Bootstrap starting.
I20260806 15:03:11.111104 4297 tablet_bootstrap.cc:654] T f53f757c60044eb5a088864867b428fc P 09573bd7d8714d91bcc69fcad505f93e: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:11.111325 4298 tablet_bootstrap.cc:654] T f53f757c60044eb5a088864867b428fc P a622511a14e14400ae2513c64e3fc5cd: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:11.111651 4299 tablet_bootstrap.cc:492] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176: Bootstrap starting.
I20260806 15:03:11.111732 4297 tablet_bootstrap.cc:492] T f53f757c60044eb5a088864867b428fc P 09573bd7d8714d91bcc69fcad505f93e: No bootstrap required, opened a new log
I20260806 15:03:11.111780 4297 ts_tablet_manager.cc:1397] T f53f757c60044eb5a088864867b428fc P 09573bd7d8714d91bcc69fcad505f93e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:11.111903 4298 tablet_bootstrap.cc:492] T f53f757c60044eb5a088864867b428fc P a622511a14e14400ae2513c64e3fc5cd: No bootstrap required, opened a new log
I20260806 15:03:11.111958 4298 ts_tablet_manager.cc:1397] T f53f757c60044eb5a088864867b428fc P a622511a14e14400ae2513c64e3fc5cd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:11.112028 4299 tablet_bootstrap.cc:654] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:11.111999 4297 raft_consensus.cc:348] T f53f757c60044eb5a088864867b428fc P 09573bd7d8714d91bcc69fcad505f93e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:11.112066 4297 raft_consensus.cc:374] T f53f757c60044eb5a088864867b428fc P 09573bd7d8714d91bcc69fcad505f93e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:11.112084 4297 raft_consensus.cc:729] T f53f757c60044eb5a088864867b428fc P 09573bd7d8714d91bcc69fcad505f93e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09573bd7d8714d91bcc69fcad505f93e, State: Initialized, Role: FOLLOWER
I20260806 15:03:11.112088 4298 raft_consensus.cc:348] T f53f757c60044eb5a088864867b428fc P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:11.112177 4298 raft_consensus.cc:374] T f53f757c60044eb5a088864867b428fc P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:11.112154 4297 consensus_queue.cc:260] T f53f757c60044eb5a088864867b428fc P 09573bd7d8714d91bcc69fcad505f93e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:11.112206 4298 raft_consensus.cc:729] T f53f757c60044eb5a088864867b428fc P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a622511a14e14400ae2513c64e3fc5cd, State: Initialized, Role: FOLLOWER
I20260806 15:03:11.112251 4298 consensus_queue.cc:260] T f53f757c60044eb5a088864867b428fc P a622511a14e14400ae2513c64e3fc5cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:11.112349 4298 ts_tablet_manager.cc:1428] T f53f757c60044eb5a088864867b428fc P a622511a14e14400ae2513c64e3fc5cd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:11.112399 4297 ts_tablet_manager.cc:1428] T f53f757c60044eb5a088864867b428fc P 09573bd7d8714d91bcc69fcad505f93e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:11.112775 4299 tablet_bootstrap.cc:492] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176: No bootstrap required, opened a new log
I20260806 15:03:11.112861 4299 ts_tablet_manager.cc:1397] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:11.113019 4299 raft_consensus.cc:348] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:11.113132 4299 raft_consensus.cc:374] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:11.113168 4299 raft_consensus.cc:729] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50556646c4e349f2a5a143e4db4c9176, State: Initialized, Role: FOLLOWER
I20260806 15:03:11.113229 4299 consensus_queue.cc:260] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:11.113400 4299 ts_tablet_manager.cc:1428] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260806 15:03:11.155790 4157 tablet.cc:2369] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:03:11.387036 4307 raft_consensus.cc:482] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:03:11.387123 4307 raft_consensus.cc:504] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:11.387437 4307 leader_election.cc:288] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a622511a14e14400ae2513c64e3fc5cd (127.0.236.65:38279), 09573bd7d8714d91bcc69fcad505f93e (127.0.236.66:39387)
I20260806 15:03:11.390952 4314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:11.387642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:46925 (local address 127.0.236.65:38279)
0806 15:03:11.387802 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:11.387804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:11.387816 (+ 12us) server_negotiation.cc:407] Connection header received
0806 15:03:11.387891 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:11.387892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:11.387904 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:11.387996 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:11.388138 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:11.388618 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:11.390025 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:11.390594 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:11.390671 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:11.390681 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:11.390835 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 15:03:11.390880 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":92,"threads_started":1}
I20260806 15:03:11.390965 4316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:11.387863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:32841 (local address 127.0.236.66:39387)
0806 15:03:11.388026 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:11.388027 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:11.388035 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:11.388058 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:11.388059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:11.388066 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:11.388121 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:11.388257 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:11.388833 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:11.390024 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:11.390594 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:11.390672 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:11.390681 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:11.390835 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 15:03:11.390873 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":89,"threads_started":1}
I20260806 15:03:11.390970 4309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:11.387582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:38279 (local address 127.0.236.67:46925)
0806 15:03:11.387755 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:11.387766 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:11.387819 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:11.388009 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:11.388010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:11.388013 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:11.388069 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:11.388071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:11.388690 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:11.388691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:11.389937 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:11.389940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:11.390675 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:11.390676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:11.390767 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:11.390771 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:03:11.390823 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 15:03:11.390874 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":16,"thread_start_us":113,"threads_started":1}
I20260806 15:03:11.390985 4315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:11.387811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:39387 (local address 127.0.236.67:32841)
0806 15:03:11.387915 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:11.387922 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:11.387969 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:11.388131 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:11.388131 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:11.388133 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:11.388201 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:11.388203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:11.388907 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:11.388908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:11.389954 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:11.389956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:11.390674 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:11.390675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:11.390774 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:11.390776 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 15:03:11.390823 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 15:03:11.390874 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":4,"thread_start_us":67,"threads_started":1}
I20260806 15:03:11.391223 3679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f53f757c60044eb5a088864867b428fc" candidate_uuid: "50556646c4e349f2a5a143e4db4c9176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622511a14e14400ae2513c64e3fc5cd" is_pre_election: true
I20260806 15:03:11.391359 3679 raft_consensus.cc:2454] T f53f757c60044eb5a088864867b428fc P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50556646c4e349f2a5a143e4db4c9176 in term 0.
I20260806 15:03:11.391376 3877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f53f757c60044eb5a088864867b428fc" candidate_uuid: "50556646c4e349f2a5a143e4db4c9176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09573bd7d8714d91bcc69fcad505f93e" is_pre_election: true
I20260806 15:03:11.391476 3877 raft_consensus.cc:2454] T f53f757c60044eb5a088864867b428fc P 09573bd7d8714d91bcc69fcad505f93e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50556646c4e349f2a5a143e4db4c9176 in term 0.
I20260806 15:03:11.391567 3987 leader_election.cc:302] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50556646c4e349f2a5a143e4db4c9176, a622511a14e14400ae2513c64e3fc5cd; no voters:
I20260806 15:03:11.391711 4307 raft_consensus.cc:2781] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:03:11.391757 4307 raft_consensus.cc:482] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:03:11.391775 4307 raft_consensus.cc:3037] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:11.392442 4307 raft_consensus.cc:504] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:11.392552 4307 leader_election.cc:288] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176 [CANDIDATE]: Term 1 election: Requested vote from peers a622511a14e14400ae2513c64e3fc5cd (127.0.236.65:38279), 09573bd7d8714d91bcc69fcad505f93e (127.0.236.66:39387)
I20260806 15:03:11.392719 3679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f53f757c60044eb5a088864867b428fc" candidate_uuid: "50556646c4e349f2a5a143e4db4c9176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a622511a14e14400ae2513c64e3fc5cd"
I20260806 15:03:11.392741 3877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f53f757c60044eb5a088864867b428fc" candidate_uuid: "50556646c4e349f2a5a143e4db4c9176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09573bd7d8714d91bcc69fcad505f93e"
I20260806 15:03:11.392789 3679 raft_consensus.cc:3037] T f53f757c60044eb5a088864867b428fc P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:11.392807 3877 raft_consensus.cc:3037] T f53f757c60044eb5a088864867b428fc P 09573bd7d8714d91bcc69fcad505f93e [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:11.393356 3877 raft_consensus.cc:2454] T f53f757c60044eb5a088864867b428fc P 09573bd7d8714d91bcc69fcad505f93e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50556646c4e349f2a5a143e4db4c9176 in term 1.
I20260806 15:03:11.393389 3679 raft_consensus.cc:2454] T f53f757c60044eb5a088864867b428fc P a622511a14e14400ae2513c64e3fc5cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50556646c4e349f2a5a143e4db4c9176 in term 1.
I20260806 15:03:11.393537 3987 leader_election.cc:302] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09573bd7d8714d91bcc69fcad505f93e, 50556646c4e349f2a5a143e4db4c9176; no voters:
I20260806 15:03:11.393622 4307 raft_consensus.cc:2781] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:11.393770 4307 raft_consensus.cc:686] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176 [term 1 LEADER]: Becoming Leader. State: Replica: 50556646c4e349f2a5a143e4db4c9176, State: Running, Role: LEADER
I20260806 15:03:11.393848 4307 consensus_queue.cc:237] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:11.394531 3436 catalog_manager.cc:5495] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50556646c4e349f2a5a143e4db4c9176 (127.0.236.67). New cstate: current_term: 1 leader_uuid: "50556646c4e349f2a5a143e4db4c9176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } health_report { overall_health: HEALTHY } } }
I20260806 15:03:11.464385 4320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:11.462278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53630 (local address 127.0.236.67:32945)
0806 15:03:11.462426 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:11.462427 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:11.462434 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:03:11.462467 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:11.462468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:11.462486 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:11.462554 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:03:11.462656 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:11.463125 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:11.463790 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:11.463985 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:11.464028 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:11.464144 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:03:11.464245 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:03:11.464293 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:11.464305 (+ 12us) server_negotiation.cc:299] Negotiation successful
0806 15:03:11.464319 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":100,"threads_started":1}
I20260806 15:03:11.468770 4314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:11.466633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:41097 (local address 127.0.236.65:38279)
0806 15:03:11.466691 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:11.466693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:11.466774 (+ 81us) server_negotiation.cc:407] Connection header received
0806 15:03:11.466828 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:11.466829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:11.466840 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:11.466880 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:03:11.466992 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:11.467433 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:11.468143 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:11.468339 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:11.468391 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:11.468558 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:03:11.468591 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:03:11.468631 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:11.468685 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 15:03:11.468703 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 15:03:11.468820 4327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:11.466555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:38279 (local address 127.0.236.67:41097)
0806 15:03:11.466718 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:11.466728 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:11.466782 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:11.466891 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:11.466891 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:11.466893 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 15:03:11.466947 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:11.466948 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:11.467504 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:11.467504 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:11.468066 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:11.468068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:11.468400 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:11.468401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:11.468474 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:11.468487 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 15:03:11.468640 (+ 153us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 15:03:11.468642 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 15:03:11.468681 (+ 39us) client_negotiation.cc:240] Negotiation successful
0806 15:03:11.468722 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
I20260806 15:03:11.470690 3877 raft_consensus.cc:1261] T f53f757c60044eb5a088864867b428fc P 09573bd7d8714d91bcc69fcad505f93e [term 1 FOLLOWER]: Refusing update from remote peer 50556646c4e349f2a5a143e4db4c9176: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:11.470784 3679 raft_consensus.cc:1261] T f53f757c60044eb5a088864867b428fc P a622511a14e14400ae2513c64e3fc5cd [term 1 FOLLOWER]: Refusing update from remote peer 50556646c4e349f2a5a143e4db4c9176: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:11.470986 4317 consensus_queue.cc:1035] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:11.471110 4307 consensus_queue.cc:1035] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:11.477545 4320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:11.475440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:41139 (local address 127.0.236.67:32945)
0806 15:03:11.475476 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:11.475477 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:11.475583 (+ 106us) server_negotiation.cc:407] Connection header received
0806 15:03:11.475623 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:11.475624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:11.475632 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:11.475671 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:03:11.475735 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:11.476158 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:11.476812 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:11.477083 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:11.477168 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:11.477325 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:03:11.477357 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:03:11.477422 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:11.477460 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 15:03:11.477479 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:03:11.477559 4335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:11.475408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:32945 (local address 127.0.236.65:41139)
0806 15:03:11.475534 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:11.475543 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:11.475589 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:11.475672 (+ 83us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:11.475672 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:11.475673 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 15:03:11.475705 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:11.475705 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:11.476204 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:11.476204 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:11.476744 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:11.476746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:11.477173 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:11.477174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:11.477236 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:11.477241 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 15:03:11.477425 (+ 184us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 15:03:11.477427 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 15:03:11.477455 (+ 28us) client_negotiation.cc:240] Negotiation successful
0806 15:03:11.477506 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":89,"threads_started":1}
I20260806 15:03:11.483816 3434 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:11.483901 3434 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:11.484486 3434 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920: Sending DeleteTablet for 3 replicas of tablet f53f757c60044eb5a088864867b428fc
I20260806 15:03:11.484740 3643 tablet_service.cc:1514] Processing DeleteTablet for tablet f53f757c60044eb5a088864867b428fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:11 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:50162
I20260806 15:03:11.484761 3847 tablet_service.cc:1514] Processing DeleteTablet for tablet f53f757c60044eb5a088864867b428fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:11 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:59658
I20260806 15:03:11.484786 4061 tablet_service.cc:1514] Processing DeleteTablet for tablet f53f757c60044eb5a088864867b428fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:11 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:53618
I20260806 15:03:11.484845 4283 tablet_replica.cc:330] T f53f757c60044eb5a088864867b428fc P a622511a14e14400ae2513c64e3fc5cd: stopping tablet replica
I20260806 15:03:11.484897 4283 raft_consensus.cc:2229] T f53f757c60044eb5a088864867b428fc P a622511a14e14400ae2513c64e3fc5cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:11.484927 4283 raft_consensus.cc:2258] T f53f757c60044eb5a088864867b428fc P a622511a14e14400ae2513c64e3fc5cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:11.484999 4281 tablet_replica.cc:330] T f53f757c60044eb5a088864867b428fc P 09573bd7d8714d91bcc69fcad505f93e: stopping tablet replica
I20260806 15:03:11.485044 4281 raft_consensus.cc:2229] T f53f757c60044eb5a088864867b428fc P 09573bd7d8714d91bcc69fcad505f93e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:11.485078 4281 raft_consensus.cc:2258] T f53f757c60044eb5a088864867b428fc P 09573bd7d8714d91bcc69fcad505f93e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:11.485146 4282 tablet_replica.cc:330] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176: stopping tablet replica
I20260806 15:03:11.485193 4282 raft_consensus.cc:2229] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176 [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:03:11.485265 4282 raft_consensus.cc:2258] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:11.485250 3434 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60340:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:03:11.485385 3434 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:03:11.485657 4282 ts_tablet_manager.cc:1905] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:11.485751 4283 ts_tablet_manager.cc:1905] T f53f757c60044eb5a088864867b428fc P a622511a14e14400ae2513c64e3fc5cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:11.485749 4281 ts_tablet_manager.cc:1905] T f53f757c60044eb5a088864867b428fc P 09573bd7d8714d91bcc69fcad505f93e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:11.487521 4283 ts_tablet_manager.cc:1918] T f53f757c60044eb5a088864867b428fc P a622511a14e14400ae2513c64e3fc5cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:03:11.487576 4283 log.cc:1196] T f53f757c60044eb5a088864867b428fc P a622511a14e14400ae2513c64e3fc5cd: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-0/wal/wals/f53f757c60044eb5a088864867b428fc
I20260806 15:03:11.487632 4281 ts_tablet_manager.cc:1918] T f53f757c60044eb5a088864867b428fc P 09573bd7d8714d91bcc69fcad505f93e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:03:11.487684 4281 log.cc:1196] T f53f757c60044eb5a088864867b428fc P 09573bd7d8714d91bcc69fcad505f93e: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-1/wal/wals/f53f757c60044eb5a088864867b428fc
I20260806 15:03:11.487820 4283 ts_tablet_manager.cc:1939] T f53f757c60044eb5a088864867b428fc P a622511a14e14400ae2513c64e3fc5cd: Deleting consensus metadata
I20260806 15:03:11.487890 4281 ts_tablet_manager.cc:1939] T f53f757c60044eb5a088864867b428fc P 09573bd7d8714d91bcc69fcad505f93e: Deleting consensus metadata
I20260806 15:03:11.488206 3417 catalog_manager.cc:4841] TS 09573bd7d8714d91bcc69fcad505f93e (127.0.236.66:39387): tablet f53f757c60044eb5a088864867b428fc (table test-table [id=aeb3e8a2e0ad49138d03d4a88fc75f14]) successfully deleted
I20260806 15:03:11.488272 3847 tablet_service.cc:1467] Processing CreateTablet for tablet c9ea39f54e7b4dd2921e2389caa0102a (DEFAULT_TABLE table=test-table [id=54a50afc34a4440fb29ead061b1ca329]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:11.488315 4061 tablet_service.cc:1467] Processing CreateTablet for tablet c9ea39f54e7b4dd2921e2389caa0102a (DEFAULT_TABLE table=test-table [id=54a50afc34a4440fb29ead061b1ca329]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:11.488369 3847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9ea39f54e7b4dd2921e2389caa0102a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:11.488623 3417 catalog_manager.cc:4841] TS a622511a14e14400ae2513c64e3fc5cd (127.0.236.65:38279): tablet f53f757c60044eb5a088864867b428fc (table test-table [id=aeb3e8a2e0ad49138d03d4a88fc75f14]) successfully deleted
I20260806 15:03:11.488638 3643 tablet_service.cc:1467] Processing CreateTablet for tablet c9ea39f54e7b4dd2921e2389caa0102a (DEFAULT_TABLE table=test-table [id=54a50afc34a4440fb29ead061b1ca329]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:11.488726 3643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9ea39f54e7b4dd2921e2389caa0102a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:11.488818 4282 ts_tablet_manager.cc:1918] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:03:11.488859 4282 log.cc:1196] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/f53f757c60044eb5a088864867b428fc
I20260806 15:03:11.489050 4282 ts_tablet_manager.cc:1939] T f53f757c60044eb5a088864867b428fc P 50556646c4e349f2a5a143e4db4c9176: Deleting consensus metadata
I20260806 15:03:11.489493 4297 tablet_bootstrap.cc:492] T c9ea39f54e7b4dd2921e2389caa0102a P 09573bd7d8714d91bcc69fcad505f93e: Bootstrap starting.
I20260806 15:03:11.489796 4297 tablet_bootstrap.cc:654] T c9ea39f54e7b4dd2921e2389caa0102a P 09573bd7d8714d91bcc69fcad505f93e: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:11.489823 4298 tablet_bootstrap.cc:492] T c9ea39f54e7b4dd2921e2389caa0102a P a622511a14e14400ae2513c64e3fc5cd: Bootstrap starting.
I20260806 15:03:11.489859 4061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9ea39f54e7b4dd2921e2389caa0102a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:11.490144 4298 tablet_bootstrap.cc:654] T c9ea39f54e7b4dd2921e2389caa0102a P a622511a14e14400ae2513c64e3fc5cd: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:11.490598 3415 catalog_manager.cc:4841] TS 50556646c4e349f2a5a143e4db4c9176 (127.0.236.67:32945): tablet f53f757c60044eb5a088864867b428fc (table test-table [id=aeb3e8a2e0ad49138d03d4a88fc75f14]) successfully deleted
I20260806 15:03:11.490952 4297 tablet_bootstrap.cc:492] T c9ea39f54e7b4dd2921e2389caa0102a P 09573bd7d8714d91bcc69fcad505f93e: No bootstrap required, opened a new log
I20260806 15:03:11.490998 4297 ts_tablet_manager.cc:1397] T c9ea39f54e7b4dd2921e2389caa0102a P 09573bd7d8714d91bcc69fcad505f93e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:03:11.491084 4299 tablet_bootstrap.cc:492] T c9ea39f54e7b4dd2921e2389caa0102a P 50556646c4e349f2a5a143e4db4c9176: Bootstrap starting.
I20260806 15:03:11.491137 4297 raft_consensus.cc:348] T c9ea39f54e7b4dd2921e2389caa0102a P 09573bd7d8714d91bcc69fcad505f93e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:11.491211 4297 raft_consensus.cc:374] T c9ea39f54e7b4dd2921e2389caa0102a P 09573bd7d8714d91bcc69fcad505f93e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:11.491228 4297 raft_consensus.cc:729] T c9ea39f54e7b4dd2921e2389caa0102a P 09573bd7d8714d91bcc69fcad505f93e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09573bd7d8714d91bcc69fcad505f93e, State: Initialized, Role: FOLLOWER
I20260806 15:03:11.491264 4297 consensus_queue.cc:260] T c9ea39f54e7b4dd2921e2389caa0102a P 09573bd7d8714d91bcc69fcad505f93e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:11.491341 4297 ts_tablet_manager.cc:1428] T c9ea39f54e7b4dd2921e2389caa0102a P 09573bd7d8714d91bcc69fcad505f93e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:11.491463 4299 tablet_bootstrap.cc:654] T c9ea39f54e7b4dd2921e2389caa0102a P 50556646c4e349f2a5a143e4db4c9176: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:11.491768 4298 tablet_bootstrap.cc:492] T c9ea39f54e7b4dd2921e2389caa0102a P a622511a14e14400ae2513c64e3fc5cd: No bootstrap required, opened a new log
I20260806 15:03:11.491819 4298 ts_tablet_manager.cc:1397] T c9ea39f54e7b4dd2921e2389caa0102a P a622511a14e14400ae2513c64e3fc5cd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:03:11.491920 4298 raft_consensus.cc:348] T c9ea39f54e7b4dd2921e2389caa0102a P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:11.492009 4298 raft_consensus.cc:374] T c9ea39f54e7b4dd2921e2389caa0102a P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:11.492029 4298 raft_consensus.cc:729] T c9ea39f54e7b4dd2921e2389caa0102a P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a622511a14e14400ae2513c64e3fc5cd, State: Initialized, Role: FOLLOWER
I20260806 15:03:11.492066 4298 consensus_queue.cc:260] T c9ea39f54e7b4dd2921e2389caa0102a P a622511a14e14400ae2513c64e3fc5cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:11.492154 4298 ts_tablet_manager.cc:1428] T c9ea39f54e7b4dd2921e2389caa0102a P a622511a14e14400ae2513c64e3fc5cd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:11.492514 4299 tablet_bootstrap.cc:492] T c9ea39f54e7b4dd2921e2389caa0102a P 50556646c4e349f2a5a143e4db4c9176: No bootstrap required, opened a new log
I20260806 15:03:11.492564 4299 ts_tablet_manager.cc:1397] T c9ea39f54e7b4dd2921e2389caa0102a P 50556646c4e349f2a5a143e4db4c9176: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:03:11.492713 4299 raft_consensus.cc:348] T c9ea39f54e7b4dd2921e2389caa0102a P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:11.492770 4299 raft_consensus.cc:374] T c9ea39f54e7b4dd2921e2389caa0102a P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:11.492789 4299 raft_consensus.cc:729] T c9ea39f54e7b4dd2921e2389caa0102a P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50556646c4e349f2a5a143e4db4c9176, State: Initialized, Role: FOLLOWER
I20260806 15:03:11.492831 4299 consensus_queue.cc:260] T c9ea39f54e7b4dd2921e2389caa0102a P 50556646c4e349f2a5a143e4db4c9176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:11.492934 4299 ts_tablet_manager.cc:1428] T c9ea39f54e7b4dd2921e2389caa0102a P 50556646c4e349f2a5a143e4db4c9176: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:11.591598 4235 raft_consensus.cc:482] T c9ea39f54e7b4dd2921e2389caa0102a P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:03:11.591701 4235 raft_consensus.cc:504] T c9ea39f54e7b4dd2921e2389caa0102a P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:11.591871 4235 leader_election.cc:288] T c9ea39f54e7b4dd2921e2389caa0102a P a622511a14e14400ae2513c64e3fc5cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09573bd7d8714d91bcc69fcad505f93e (127.0.236.66:39387), 50556646c4e349f2a5a143e4db4c9176 (127.0.236.67:32945)
I20260806 15:03:11.592049 4087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9ea39f54e7b4dd2921e2389caa0102a" candidate_uuid: "a622511a14e14400ae2513c64e3fc5cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50556646c4e349f2a5a143e4db4c9176" is_pre_election: true
I20260806 15:03:11.592060 3877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9ea39f54e7b4dd2921e2389caa0102a" candidate_uuid: "a622511a14e14400ae2513c64e3fc5cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09573bd7d8714d91bcc69fcad505f93e" is_pre_election: true
I20260806 15:03:11.592129 4087 raft_consensus.cc:2454] T c9ea39f54e7b4dd2921e2389caa0102a P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a622511a14e14400ae2513c64e3fc5cd in term 0.
I20260806 15:03:11.592154 3877 raft_consensus.cc:2454] T c9ea39f54e7b4dd2921e2389caa0102a P 09573bd7d8714d91bcc69fcad505f93e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a622511a14e14400ae2513c64e3fc5cd in term 0.
I20260806 15:03:11.592291 3577 leader_election.cc:302] T c9ea39f54e7b4dd2921e2389caa0102a P a622511a14e14400ae2513c64e3fc5cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50556646c4e349f2a5a143e4db4c9176, a622511a14e14400ae2513c64e3fc5cd; no voters:
I20260806 15:03:11.592392 4235 raft_consensus.cc:2781] T c9ea39f54e7b4dd2921e2389caa0102a P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:03:11.592453 4235 raft_consensus.cc:482] T c9ea39f54e7b4dd2921e2389caa0102a P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:03:11.592468 4235 raft_consensus.cc:3037] T c9ea39f54e7b4dd2921e2389caa0102a P a622511a14e14400ae2513c64e3fc5cd [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:11.593154 4235 raft_consensus.cc:504] T c9ea39f54e7b4dd2921e2389caa0102a P a622511a14e14400ae2513c64e3fc5cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:11.593272 4235 leader_election.cc:288] T c9ea39f54e7b4dd2921e2389caa0102a P a622511a14e14400ae2513c64e3fc5cd [CANDIDATE]: Term 1 election: Requested vote from peers 09573bd7d8714d91bcc69fcad505f93e (127.0.236.66:39387), 50556646c4e349f2a5a143e4db4c9176 (127.0.236.67:32945)
I20260806 15:03:11.593400 3877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9ea39f54e7b4dd2921e2389caa0102a" candidate_uuid: "a622511a14e14400ae2513c64e3fc5cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09573bd7d8714d91bcc69fcad505f93e"
I20260806 15:03:11.593417 4087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9ea39f54e7b4dd2921e2389caa0102a" candidate_uuid: "a622511a14e14400ae2513c64e3fc5cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50556646c4e349f2a5a143e4db4c9176"
I20260806 15:03:11.593477 4087 raft_consensus.cc:3037] T c9ea39f54e7b4dd2921e2389caa0102a P 50556646c4e349f2a5a143e4db4c9176 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:11.593479 3877 raft_consensus.cc:3037] T c9ea39f54e7b4dd2921e2389caa0102a P 09573bd7d8714d91bcc69fcad505f93e [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:11.594023 4087 raft_consensus.cc:2454] T c9ea39f54e7b4dd2921e2389caa0102a P 50556646c4e349f2a5a143e4db4c9176 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622511a14e14400ae2513c64e3fc5cd in term 1.
I20260806 15:03:11.594156 3877 raft_consensus.cc:2454] T c9ea39f54e7b4dd2921e2389caa0102a P 09573bd7d8714d91bcc69fcad505f93e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a622511a14e14400ae2513c64e3fc5cd in term 1.
I20260806 15:03:11.594180 3577 leader_election.cc:302] T c9ea39f54e7b4dd2921e2389caa0102a P a622511a14e14400ae2513c64e3fc5cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50556646c4e349f2a5a143e4db4c9176, a622511a14e14400ae2513c64e3fc5cd; no voters:
I20260806 15:03:11.594257 4235 raft_consensus.cc:2781] T c9ea39f54e7b4dd2921e2389caa0102a P a622511a14e14400ae2513c64e3fc5cd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:11.594302 4235 raft_consensus.cc:686] T c9ea39f54e7b4dd2921e2389caa0102a P a622511a14e14400ae2513c64e3fc5cd [term 1 LEADER]: Becoming Leader. State: Replica: a622511a14e14400ae2513c64e3fc5cd, State: Running, Role: LEADER
I20260806 15:03:11.594347 4235 consensus_queue.cc:237] T c9ea39f54e7b4dd2921e2389caa0102a P a622511a14e14400ae2513c64e3fc5cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } }
I20260806 15:03:11.594882 3434 catalog_manager.cc:5495] T c9ea39f54e7b4dd2921e2389caa0102a P a622511a14e14400ae2513c64e3fc5cd reported cstate change: term changed from 0 to 1, leader changed from <none> to a622511a14e14400ae2513c64e3fc5cd (127.0.236.65). New cstate: current_term: 1 leader_uuid: "a622511a14e14400ae2513c64e3fc5cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a622511a14e14400ae2513c64e3fc5cd" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 } health_report { overall_health: UNKNOWN } } }
I20260806 15:03:11.608222 4314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:11.606037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50188 (local address 127.0.236.65:38279)
0806 15:03:11.606083 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:11.606085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:11.606144 (+ 59us) server_negotiation.cc:407] Connection header received
0806 15:03:11.606169 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:11.606170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:11.606180 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:11.606231 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:03:11.606354 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:11.606990 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:11.607732 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:11.607930 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:11.607981 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:11.608061 (+ 80us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:03:11.608086 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:03:11.608115 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:11.608145 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 15:03:11.608159 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 15:03:11.610059 3877 raft_consensus.cc:1261] T c9ea39f54e7b4dd2921e2389caa0102a P 09573bd7d8714d91bcc69fcad505f93e [term 1 FOLLOWER]: Refusing update from remote peer a622511a14e14400ae2513c64e3fc5cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:11.610066 4087 raft_consensus.cc:1261] T c9ea39f54e7b4dd2921e2389caa0102a P 50556646c4e349f2a5a143e4db4c9176 [term 1 FOLLOWER]: Refusing update from remote peer a622511a14e14400ae2513c64e3fc5cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:11.610248 4257 consensus_queue.cc:1035] T c9ea39f54e7b4dd2921e2389caa0102a P a622511a14e14400ae2513c64e3fc5cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "09573bd7d8714d91bcc69fcad505f93e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 39387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:11.610342 4344 consensus_queue.cc:1035] T c9ea39f54e7b4dd2921e2389caa0102a P a622511a14e14400ae2513c64e3fc5cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "50556646c4e349f2a5a143e4db4c9176" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 32945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 15:03:11.614235 4255 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 15:03:13.129329 3434 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:13.129431 3434 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:13.130234 3434 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0dba63e943394e3ab6a9a29709510920: Sending DeleteTablet for 3 replicas of tablet c9ea39f54e7b4dd2921e2389caa0102a
I20260806 15:03:13.130543 4061 tablet_service.cc:1514] Processing DeleteTablet for tablet c9ea39f54e7b4dd2921e2389caa0102a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:13 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:53618
I20260806 15:03:13.130543 3847 tablet_service.cc:1514] Processing DeleteTablet for tablet c9ea39f54e7b4dd2921e2389caa0102a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:13 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:59658
I20260806 15:03:13.130631 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 3552
I20260806 15:03:13.130718 3644 tablet_service.cc:1514] Processing DeleteTablet for tablet c9ea39f54e7b4dd2921e2389caa0102a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:13 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:50162
I20260806 15:03:13.136215 4375 tablet_replica.cc:330] T c9ea39f54e7b4dd2921e2389caa0102a P 09573bd7d8714d91bcc69fcad505f93e: stopping tablet replica
I20260806 15:03:13.136279 4375 raft_consensus.cc:2229] T c9ea39f54e7b4dd2921e2389caa0102a P 09573bd7d8714d91bcc69fcad505f93e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:13.136319 4375 raft_consensus.cc:2258] T c9ea39f54e7b4dd2921e2389caa0102a P 09573bd7d8714d91bcc69fcad505f93e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:13.136445 4376 tablet_replica.cc:330] T c9ea39f54e7b4dd2921e2389caa0102a P 50556646c4e349f2a5a143e4db4c9176: stopping tablet replica
I20260806 15:03:13.136530 4376 raft_consensus.cc:2229] T c9ea39f54e7b4dd2921e2389caa0102a P 50556646c4e349f2a5a143e4db4c9176 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:13.136576 4376 raft_consensus.cc:2258] T c9ea39f54e7b4dd2921e2389caa0102a P 50556646c4e349f2a5a143e4db4c9176 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:13.136590 4375 ts_tablet_manager.cc:1905] T c9ea39f54e7b4dd2921e2389caa0102a P 09573bd7d8714d91bcc69fcad505f93e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:13.136802 4376 ts_tablet_manager.cc:1905] T c9ea39f54e7b4dd2921e2389caa0102a P 50556646c4e349f2a5a143e4db4c9176: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:13.138226 4375 ts_tablet_manager.cc:1918] T c9ea39f54e7b4dd2921e2389caa0102a P 09573bd7d8714d91bcc69fcad505f93e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 15:03:13.138293 4375 log.cc:1196] T c9ea39f54e7b4dd2921e2389caa0102a P 09573bd7d8714d91bcc69fcad505f93e: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-1/wal/wals/c9ea39f54e7b4dd2921e2389caa0102a
I20260806 15:03:13.138326 4376 ts_tablet_manager.cc:1918] T c9ea39f54e7b4dd2921e2389caa0102a P 50556646c4e349f2a5a143e4db4c9176: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 15:03:13.138427 4376 log.cc:1196] T c9ea39f54e7b4dd2921e2389caa0102a P 50556646c4e349f2a5a143e4db4c9176: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/c9ea39f54e7b4dd2921e2389caa0102a
I20260806 15:03:13.138510 4375 ts_tablet_manager.cc:1939] T c9ea39f54e7b4dd2921e2389caa0102a P 09573bd7d8714d91bcc69fcad505f93e: Deleting consensus metadata
I20260806 15:03:13.138638 4376 ts_tablet_manager.cc:1939] T c9ea39f54e7b4dd2921e2389caa0102a P 50556646c4e349f2a5a143e4db4c9176: Deleting consensus metadata
I20260806 15:03:13.138815 3417 catalog_manager.cc:4841] TS 09573bd7d8714d91bcc69fcad505f93e (127.0.236.66:39387): tablet c9ea39f54e7b4dd2921e2389caa0102a (table test-table [id=54a50afc34a4440fb29ead061b1ca329]) successfully deleted
I20260806 15:03:13.138906 3415 catalog_manager.cc:4841] TS 50556646c4e349f2a5a143e4db4c9176 (127.0.236.67:32945): tablet c9ea39f54e7b4dd2921e2389caa0102a (table test-table [id=54a50afc34a4440fb29ead061b1ca329]) successfully deleted
W20260806 15:03:13.142026 3417 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.236.65:38279 (error 108)
I20260806 15:03:13.142566 4379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:13.142382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:38279 (local address 127.0.0.1:54682)
0806 15:03:13.142494 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:13.142534 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.65:38279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":67,"threads_started":1}
I20260806 15:03:13.142583 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 3754
W20260806 15:03:13.142681 3417 catalog_manager.cc:4584] TS a622511a14e14400ae2513c64e3fc5cd (127.0.236.65:38279): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c9ea39f54e7b4dd2921e2389caa0102a: Network error: Client connection negotiation failed: client connection to 127.0.236.65:38279: connect: Connection refused (error 111)
I20260806 15:03:13.162034 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 3959
I20260806 15:03:13.171531 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 3393
2026-08-06T15:03:13Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5833 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:03:13 dist-test-slave-tc2s krb5kdc[4382](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:03:13 dist-test-slave-tc2s krb5kdc[4382](info): set up 2 sockets
Aug 06 15:03:13 dist-test-slave-tc2s krb5kdc[4382](info): commencing operation
krb5kdc: starting...
W20260806 15:03:14.287935 945 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:03:14 dist-test-slave-tc2s krb5kdc[4382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:03:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:03:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:03:14.394207 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:44241
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:38159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:44241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:14.472570 4399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:14.472756 4399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:14.472775 4399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:14.473922 4399 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:03:14.473960 4399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:14.473973 4399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:14.473994 4399 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:03:14.474017 4399 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:03:14.475495 4399 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:38159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:44241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:44241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:14.475728 4399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:14.475925 4399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 15:03:14.480569 4399 server_base.cc:1034] running on GCE node
W20260806 15:03:14.480543 4411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:14.480630 4408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:14.480527 4409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:14.481020 4399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:14.481288 4399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:14.482429 4399 hybrid_clock.cc:648] HybridClock initialized: now 1786028594482413 us; error 28 us; skew 500 ppm
Aug 06 15:03:14 dist-test-slave-tc2s krb5kdc[4382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028594, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:14.484658 4399 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:03:14.485078 4399 webserver.cc:460] Webserver started at http://127.0.236.126:34899/ using document root <none> and password file <none>
I20260806 15:03:14.485280 4399 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:14.485325 4399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:14.485456 4399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:14.486483 4399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "5286c1ec972c4103913368667281ea7f"
format_stamp: "Formatted at 2026-08-06 15:03:14 on dist-test-slave-tc2s"
server_key: "50e40c696d8bdeecbed46e7e5435184e"
server_key_iv: "3390a8b923e7ee4585adb0baa636e963"
server_key_version: "encryptionkey@0"
I20260806 15:03:14.486794 4399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "5286c1ec972c4103913368667281ea7f"
format_stamp: "Formatted at 2026-08-06 15:03:14 on dist-test-slave-tc2s"
server_key: "50e40c696d8bdeecbed46e7e5435184e"
server_key_iv: "3390a8b923e7ee4585adb0baa636e963"
server_key_version: "encryptionkey@0"
I20260806 15:03:14.487979 4399 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:03:14.488603 4421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:14.488745 4399 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:14.488829 4399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "5286c1ec972c4103913368667281ea7f"
format_stamp: "Formatted at 2026-08-06 15:03:14 on dist-test-slave-tc2s"
server_key: "50e40c696d8bdeecbed46e7e5435184e"
server_key_iv: "3390a8b923e7ee4585adb0baa636e963"
server_key_version: "encryptionkey@0"
I20260806 15:03:14.488888 4399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:14.508100 4399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:14.509661 4399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:14.509786 4399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:14.513522 4399 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:44241
I20260806 15:03:14.513561 4499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:44241 every 8 connection(s)
I20260806 15:03:14.514078 4399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:03:14.514695 4501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:14.516770 4501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5286c1ec972c4103913368667281ea7f: Bootstrap starting.
I20260806 15:03:14.517321 4501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5286c1ec972c4103913368667281ea7f: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:14.517593 4501 log.cc:824] T 00000000000000000000000000000000 P 5286c1ec972c4103913368667281ea7f: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:14.518169 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 4399
I20260806 15:03:14.518244 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:03:14.518246 4501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5286c1ec972c4103913368667281ea7f: No bootstrap required, opened a new log
I20260806 15:03:14.518386 945 external_mini_cluster.cc:1428] Setting key 7ace264347a1f4c694fe44547e1f3264
I20260806 15:03:14.520922 4501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5286c1ec972c4103913368667281ea7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5286c1ec972c4103913368667281ea7f" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44241 } }
I20260806 15:03:14.521080 4501 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5286c1ec972c4103913368667281ea7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:14.521111 4501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5286c1ec972c4103913368667281ea7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5286c1ec972c4103913368667281ea7f, State: Initialized, Role: FOLLOWER
I20260806 15:03:14.521221 4501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5286c1ec972c4103913368667281ea7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5286c1ec972c4103913368667281ea7f" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44241 } }
I20260806 15:03:14.521283 4501 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5286c1ec972c4103913368667281ea7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:03:14.521314 4501 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5286c1ec972c4103913368667281ea7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:03:14.521353 4501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5286c1ec972c4103913368667281ea7f [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:14.521999 4501 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5286c1ec972c4103913368667281ea7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5286c1ec972c4103913368667281ea7f" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44241 } }
I20260806 15:03:14.522105 4501 leader_election.cc:302] T 00000000000000000000000000000000 P 5286c1ec972c4103913368667281ea7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5286c1ec972c4103913368667281ea7f; no voters:
I20260806 15:03:14.522305 4501 leader_election.cc:288] T 00000000000000000000000000000000 P 5286c1ec972c4103913368667281ea7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:03:14.522466 4508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5286c1ec972c4103913368667281ea7f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:14.522562 4501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5286c1ec972c4103913368667281ea7f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:03:14.522663 4508 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5286c1ec972c4103913368667281ea7f [term 1 LEADER]: Becoming Leader. State: Replica: 5286c1ec972c4103913368667281ea7f, State: Running, Role: LEADER
I20260806 15:03:14.522763 4508 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5286c1ec972c4103913368667281ea7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5286c1ec972c4103913368667281ea7f" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44241 } }
I20260806 15:03:14.523061 4508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5286c1ec972c4103913368667281ea7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5286c1ec972c4103913368667281ea7f. Latest consensus state: current_term: 1 leader_uuid: "5286c1ec972c4103913368667281ea7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5286c1ec972c4103913368667281ea7f" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44241 } } }
I20260806 15:03:14.523129 4508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5286c1ec972c4103913368667281ea7f [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:14.523286 4507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5286c1ec972c4103913368667281ea7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5286c1ec972c4103913368667281ea7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5286c1ec972c4103913368667281ea7f" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44241 } } }
I20260806 15:03:14.523502 4507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5286c1ec972c4103913368667281ea7f [sys.catalog]: This master's current role is: LEADER
W20260806 15:03:14.524621 4532 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5286c1ec972c4103913368667281ea7f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:03:14.524679 4532 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:03:14.524725 4513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:03:14.524887 4513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 15:03:14 dist-test-slave-tc2s krb5kdc[4382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:14.526360 4513 catalog_manager.cc:1347] Generated new cluster ID: eb822574857f40d8b3ca2554bc873e6e
I20260806 15:03:14.526413 4513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:03:14.528961 4506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:14.518861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37916 (local address 127.0.236.126:44241)
0806 15:03:14.519028 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:14.519037 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:14.519070 (+ 33us) server_negotiation.cc:407] Connection header received
0806 15:03:14.519475 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:14.519484 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:14.519542 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:14.519684 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:14.520238 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:14.521543 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:14.522578 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:14.523167 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:14.523273 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:14.525942 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:14.525946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:14.525946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:14.525958 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:14.527658 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:14.527963 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:14.527966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:14.527966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:14.528061 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:14.528196 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:14.528196 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:14.528197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:14.528449 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:14.528559 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:14.528645 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 15:03:14.528765 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":75,"threads_started":1}
I20260806 15:03:14.531685 4513 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:03:14.532155 4513 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:03:14.535768 4513 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5286c1ec972c4103913368667281ea7f: Generated new TSK 0
I20260806 15:03:14.535928 4513 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.236.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
I20260806 15:03:14.562490 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:0
--local_ip_for_outbound_sockets=127.0.236.65
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44241
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:38159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:14.641376 4540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:14.641570 4540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:14.641592 4540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:14.642729 4540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:14.642769 4540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:14.642794 4540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:03:14.644235 4540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:38159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:14.644441 4540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:14.644635 4540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:14.650148 4559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:14.650127 4556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:14.650110 4554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:14.650345 4540 server_base.cc:1034] running on GCE node
I20260806 15:03:14.650581 4540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:14.650799 4540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:14.651947 4540 hybrid_clock.cc:648] HybridClock initialized: now 1786028594651936 us; error 29 us; skew 500 ppm
Aug 06 15:03:14 dist-test-slave-tc2s krb5kdc[4382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028594, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:14.654265 4540 init.cc:377] Logged in from keytab as kudu/127.0.236.65@KRBTEST.COM (short username kudu)
I20260806 15:03:14.654647 4540 webserver.cc:460] Webserver started at http://127.0.236.65:39109/ using document root <none> and password file <none>
I20260806 15:03:14.654827 4540 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:14.654873 4540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:14.654997 4540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:14.655931 4540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/data/instance:
uuid: "ef595dedfc3d49e7a417c507c2e6a666"
format_stamp: "Formatted at 2026-08-06 15:03:14 on dist-test-slave-tc2s"
server_key: "4a0878582f2c5f3315483a0b2f9e41b5"
server_key_iv: "097763c6166bb4f33857d032dc84367e"
server_key_version: "encryptionkey@0"
I20260806 15:03:14.656253 4540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/wal/instance:
uuid: "ef595dedfc3d49e7a417c507c2e6a666"
format_stamp: "Formatted at 2026-08-06 15:03:14 on dist-test-slave-tc2s"
server_key: "4a0878582f2c5f3315483a0b2f9e41b5"
server_key_iv: "097763c6166bb4f33857d032dc84367e"
server_key_version: "encryptionkey@0"
I20260806 15:03:14.657428 4540 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 15:03:14.658088 4569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:14.658236 4540 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:14.658347 4540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "ef595dedfc3d49e7a417c507c2e6a666"
format_stamp: "Formatted at 2026-08-06 15:03:14 on dist-test-slave-tc2s"
server_key: "4a0878582f2c5f3315483a0b2f9e41b5"
server_key_iv: "097763c6166bb4f33857d032dc84367e"
server_key_version: "encryptionkey@0"
I20260806 15:03:14.658411 4540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:14.687592 4540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:14.689193 4540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:14.689317 4540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:14.689527 4540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:14.689800 4540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:14.689836 4540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:14.689864 4540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:14.689908 4540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:14.695209 4540 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:38677
I20260806 15:03:14.695268 4732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:38677 every 8 connection(s)
I20260806 15:03:14.695530 4540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:03:14.696704 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 4540
I20260806 15:03:14.696789 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/wal/instance
I20260806 15:03:14.696956 945 external_mini_cluster.cc:1428] Setting key 60225272050675193f62102105b46b9f
Aug 06 15:03:14 dist-test-slave-tc2s krb5kdc[4382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028594, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:14.704824 4506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:14.696148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:41209 (local address 127.0.236.126:44241)
0806 15:03:14.696218 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:14.696221 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:14.696351 (+ 130us) server_negotiation.cc:407] Connection header received
0806 15:03:14.697283 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:14.697284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:14.697293 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:14.697373 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:14.698025 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:14.698693 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:14.699924 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:14.700134 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:14.700187 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:14.702360 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:14.702362 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:14.702363 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:14.702369 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:14.703591 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:14.703960 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:14.703961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:14.703961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:14.704015 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:14.704230 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:14.704231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:14.704231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:14.704336 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:14.704393 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:14.704650 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 15:03:14.704735 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 15:03:14.704954 4735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:14.696122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:44241 (local address 127.0.236.65:41209)
0806 15:03:14.696251 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:14.696284 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:14.697192 (+ 908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:14.697437 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:14.697441 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:14.697750 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:14.697948 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:14.697953 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:14.698769 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:14.698770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:14.699840 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:14.699842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:14.700211 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:14.700213 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:14.700359 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:14.700816 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:14.700825 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:14.702269 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:14.703666 (+ 1397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:14.703674 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:14.703676 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:14.703885 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:14.704070 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:14.704070 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:14.704071 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:14.704174 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:14.704392 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:14.704399 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:14.704533 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 15:03:14.704616 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 15:03:14.704760 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for kudu/127.0.236.66@KRBTEST.COM; defaulting to no policy
I20260806 15:03:14.705565 4733 heartbeater.cc:347] Connected to a master server at 127.0.236.126:44241
I20260806 15:03:14.705657 4733 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:14.705789 4733 heartbeater.cc:511] Master 127.0.236.126:44241 requested a full tablet report, sending...
I20260806 15:03:14.706274 4455 ts_manager.cc:194] Registered new tserver with Master: ef595dedfc3d49e7a417c507c2e6a666 (127.0.236.65:38677)
I20260806 15:03:14.706836 4455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.65@KRBTEST.COM'} at 127.0.236.65:41209
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
I20260806 15:03:14.720063 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:0
--local_ip_for_outbound_sockets=127.0.236.66
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44241
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:38159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:14.797441 4740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:14.797618 4740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:14.797640 4740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:14.798745 4740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:14.798787 4740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:14.798815 4740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:03:14.800218 4740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:38159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:14.800388 4740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:14.800549 4740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:14.804975 4748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:14.804972 4746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:14.804997 4750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:14.805162 4740 server_base.cc:1034] running on GCE node
I20260806 15:03:14.805353 4740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:14.805562 4740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:14.806690 4740 hybrid_clock.cc:648] HybridClock initialized: now 1786028594806673 us; error 31 us; skew 500 ppm
Aug 06 15:03:14 dist-test-slave-tc2s krb5kdc[4382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028594, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:14.809021 4740 init.cc:377] Logged in from keytab as kudu/127.0.236.66@KRBTEST.COM (short username kudu)
I20260806 15:03:14.809374 4740 webserver.cc:460] Webserver started at http://127.0.236.66:34117/ using document root <none> and password file <none>
I20260806 15:03:14.809537 4740 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:14.809574 4740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:14.809660 4740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:14.810590 4740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/data/instance:
uuid: "305bcbd207d144968c4c493bf4f8a99e"
format_stamp: "Formatted at 2026-08-06 15:03:14 on dist-test-slave-tc2s"
server_key: "bcb80826fc6bbfa22d35660216de8956"
server_key_iv: "c65ced1fc17265ad79b1c22097255eaf"
server_key_version: "encryptionkey@0"
I20260806 15:03:14.810861 4740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/wal/instance:
uuid: "305bcbd207d144968c4c493bf4f8a99e"
format_stamp: "Formatted at 2026-08-06 15:03:14 on dist-test-slave-tc2s"
server_key: "bcb80826fc6bbfa22d35660216de8956"
server_key_iv: "c65ced1fc17265ad79b1c22097255eaf"
server_key_version: "encryptionkey@0"
I20260806 15:03:14.811977 4740 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:03:14.812566 4764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:14.812701 4740 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:14.812781 4740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "305bcbd207d144968c4c493bf4f8a99e"
format_stamp: "Formatted at 2026-08-06 15:03:14 on dist-test-slave-tc2s"
server_key: "bcb80826fc6bbfa22d35660216de8956"
server_key_iv: "c65ced1fc17265ad79b1c22097255eaf"
server_key_version: "encryptionkey@0"
I20260806 15:03:14.812831 4740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:14.824019 4740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:14.825955 4740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:14.826057 4740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:14.826221 4740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:14.826471 4740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:14.826503 4740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:14.826529 4740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:14.826541 4740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:14.831568 4740 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:37845
I20260806 15:03:14.831627 4943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:37845 every 8 connection(s)
I20260806 15:03:14.831871 4740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
I20260806 15:03:14.833989 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 4740
I20260806 15:03:14.834082 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/wal/instance
I20260806 15:03:14.834290 945 external_mini_cluster.cc:1428] Setting key 9692220cd6419588071f4c283cf4a37c
Aug 06 15:03:14 dist-test-slave-tc2s krb5kdc[4382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028594, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:14.840607 4506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:14.832496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:56449 (local address 127.0.236.126:44241)
0806 15:03:14.832573 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:14.832575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:14.832735 (+ 160us) server_negotiation.cc:407] Connection header received
0806 15:03:14.833556 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:14.833558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:14.833570 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:14.833629 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:14.834395 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:14.835097 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:14.836023 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:14.836226 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:14.836281 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:14.838171 (+ 1890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:14.838172 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:14.838173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:14.838180 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:14.839388 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:14.839741 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:14.839742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:14.839743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:14.839798 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:14.840014 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:14.840015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:14.840015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:14.840124 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:14.840178 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:14.840436 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 15:03:14.840519 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 15:03:14.840729 4948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:14.832453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:44241 (local address 127.0.236.66:56449)
0806 15:03:14.832618 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:14.832660 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:14.833467 (+ 807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:14.833693 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:14.833696 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:14.834037 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:14.834289 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:14.834295 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:14.835176 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:14.835177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:14.835947 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:14.835950 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:14.836291 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:14.836292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:14.836411 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:14.836812 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:14.836820 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:14.838092 (+ 1272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:14.839467 (+ 1375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:14.839473 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:14.839475 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:14.839678 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:14.839851 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:14.839851 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:14.839851 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:14.839959 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:14.840177 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:14.840183 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:14.840304 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 15:03:14.840397 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 15:03:14.840538 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
I20260806 15:03:14.841372 4946 heartbeater.cc:347] Connected to a master server at 127.0.236.126:44241
I20260806 15:03:14.841452 4946 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:14.841606 4946 heartbeater.cc:511] Master 127.0.236.126:44241 requested a full tablet report, sending...
I20260806 15:03:14.842007 4455 ts_manager.cc:194] Registered new tserver with Master: 305bcbd207d144968c4c493bf4f8a99e (127.0.236.66:37845)
WARNING: no policy specified for kudu/127.0.236.67@KRBTEST.COM; defaulting to no policy
I20260806 15:03:14.842389 4455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.66@KRBTEST.COM'} at 127.0.236.66:56449
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
I20260806 15:03:14.857451 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:0
--local_ip_for_outbound_sockets=127.0.236.67
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44241
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:38159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:14.936158 4953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:14.936343 4953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:14.936364 4953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:14.937456 4953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:14.937489 4953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:14.937512 4953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:03:14.938988 4953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:38159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:14.939158 4953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:14.939327 4953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:14.943895 4963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:14.943893 4970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:14.943922 4953 server_base.cc:1034] running on GCE node
W20260806 15:03:14.943893 4962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:14.944270 4953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:14.944481 4953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:14.945614 4953 hybrid_clock.cc:648] HybridClock initialized: now 1786028594945598 us; error 29 us; skew 500 ppm
Aug 06 15:03:14 dist-test-slave-tc2s krb5kdc[4382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028594, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:14.947904 4953 init.cc:377] Logged in from keytab as kudu/127.0.236.67@KRBTEST.COM (short username kudu)
I20260806 15:03:14.948312 4953 webserver.cc:460] Webserver started at http://127.0.236.67:46745/ using document root <none> and password file <none>
I20260806 15:03:14.948499 4953 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:14.948544 4953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:14.948662 4953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:14.949581 4953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/data/instance:
uuid: "32ef9df019534a939f159d3261aa8a5b"
format_stamp: "Formatted at 2026-08-06 15:03:14 on dist-test-slave-tc2s"
server_key: "461fcde24db34a497cf7bf6cb020c819"
server_key_iv: "4b255739d5c7f64fca44b1daa0b2c16c"
server_key_version: "encryptionkey@0"
I20260806 15:03:14.950016 4953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/wal/instance:
uuid: "32ef9df019534a939f159d3261aa8a5b"
format_stamp: "Formatted at 2026-08-06 15:03:14 on dist-test-slave-tc2s"
server_key: "461fcde24db34a497cf7bf6cb020c819"
server_key_iv: "4b255739d5c7f64fca44b1daa0b2c16c"
server_key_version: "encryptionkey@0"
I20260806 15:03:14.951208 4953 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:14.951857 4981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:14.952004 4953 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:14.952085 4953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "32ef9df019534a939f159d3261aa8a5b"
format_stamp: "Formatted at 2026-08-06 15:03:14 on dist-test-slave-tc2s"
server_key: "461fcde24db34a497cf7bf6cb020c819"
server_key_iv: "4b255739d5c7f64fca44b1daa0b2c16c"
server_key_version: "encryptionkey@0"
I20260806 15:03:14.952148 4953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:14.975152 4953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:14.976702 4953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:14.976835 4953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:14.977029 4953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:14.977299 4953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:14.977331 4953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:14.977357 4953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:14.977375 4953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:14.982648 4953 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:34035
I20260806 15:03:14.982682 5139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:34035 every 8 connection(s)
I20260806 15:03:14.982955 4953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
Aug 06 15:03:14 dist-test-slave-tc2s krb5kdc[4382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028594, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:14.990903 4506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:14.983571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:45797 (local address 127.0.236.126:44241)
0806 15:03:14.983629 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:14.983632 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:14.983811 (+ 179us) server_negotiation.cc:407] Connection header received
0806 15:03:14.984635 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:14.984636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:14.984646 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:14.984693 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:14.985208 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:14.985697 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:14.986618 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:14.986795 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:14.986842 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:14.988598 (+ 1756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:14.988600 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:14.988600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:14.988608 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:14.989792 (+ 1184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:14.990181 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:14.990181 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:14.990182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:14.990230 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:14.990439 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:14.990439 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:14.990439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:14.990530 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:14.990571 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:14.990784 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 15:03:14.990839 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 15:03:14.991114 5146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:14.983537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:44241 (local address 127.0.236.67:45797)
0806 15:03:14.983707 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:14.983740 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:14.984556 (+ 816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:14.984741 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:14.984744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:14.984977 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:14.985150 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:14.985154 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:14.985754 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:14.985755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:14.986555 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:14.986557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:14.986850 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:14.986850 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:14.986946 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:14.987298 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:14.987306 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:14.988500 (+ 1194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:14.989850 (+ 1350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:14.989852 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:14.989854 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:14.990122 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:14.990273 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:14.990274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:14.990274 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:14.990395 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:14.990570 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:14.990573 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:14.990687 (+ 114us) client_negotiation.cc:769] Sending connection context
0806 15:03:14.990771 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 15:03:14.990881 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":98,"threads_started":1}
I20260806 15:03:14.991542 5144 heartbeater.cc:347] Connected to a master server at 127.0.236.126:44241
I20260806 15:03:14.991621 5144 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:14.991773 5144 heartbeater.cc:511] Master 127.0.236.126:44241 requested a full tablet report, sending...
I20260806 15:03:14.991792 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 4953
I20260806 15:03:14.991876 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/wal/instance
I20260806 15:03:14.992060 945 external_mini_cluster.cc:1428] Setting key 6c35e7c86799606356dd95469a0ae233
I20260806 15:03:14.992144 4455 ts_manager.cc:194] Registered new tserver with Master: 32ef9df019534a939f159d3261aa8a5b (127.0.236.67:34035)
I20260806 15:03:14.992601 4455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.67@KRBTEST.COM'} at 127.0.236.67:45797
I20260806 15:03:14.992918 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 15:03:15.004736 4506 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:15.003990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37930 (local address 127.0.236.126:44241)
0806 15:03:15.004044 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:15.004046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:15.004142 (+ 96us) server_negotiation.cc:407] Connection header received
0806 15:03:15.004192 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:15.004192 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:15.004201 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:15.004235 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:15.004678 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37930: BlockingRecv error: recv got EOF from 127.0.0.1:37930 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 15:03:15.005587 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 4540
I20260806 15:03:15.015733 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 4740
I20260806 15:03:15.031997 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 4953
I20260806 15:03:15.047822 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 4399
2026-08-06T15:03:15Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1871 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:03:15 dist-test-slave-tc2s krb5kdc[5164](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:03:15 dist-test-slave-tc2s krb5kdc[5164](info): set up 2 sockets
Aug 06 15:03:15 dist-test-slave-tc2s krb5kdc[5164](info): commencing operation
krb5kdc: starting...
W20260806 15:03:16.159125 945 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:03:16 dist-test-slave-tc2s krb5kdc[5164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:03:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:03:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:03:16.266063 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:45911
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:46759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:45911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:16.351133 5180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:16.351397 5180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:16.351432 5180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:16.353091 5180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:03:16.353145 5180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:16.353168 5180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:16.353184 5180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:03:16.353209 5180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:03:16.355301 5180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:46759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:45911
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:45911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:16.357105 5180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:16.357334 5180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:16.362126 5188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:16.362142 5195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:16.362334 5180 server_base.cc:1034] running on GCE node
W20260806 15:03:16.362128 5190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:16.362703 5180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:16.363029 5180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:16.364185 5180 hybrid_clock.cc:648] HybridClock initialized: now 1786028596364168 us; error 32 us; skew 500 ppm
Aug 06 15:03:16 dist-test-slave-tc2s krb5kdc[5164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028596, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:16.367240 5180 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:03:16.367691 5180 webserver.cc:460] Webserver started at http://127.0.236.126:44151/ using document root <none> and password file <none>
I20260806 15:03:16.367915 5180 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:16.367969 5180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:16.368100 5180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:16.369478 5180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "e5b91fff821d431f99f1ced497469760"
format_stamp: "Formatted at 2026-08-06 15:03:16 on dist-test-slave-tc2s"
server_key: "a069f44b7ae2222d6392681e62d3774f"
server_key_iv: "864c63f7292b78a1d4e897994df8d159"
server_key_version: "encryptionkey@0"
I20260806 15:03:16.369863 5180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "e5b91fff821d431f99f1ced497469760"
format_stamp: "Formatted at 2026-08-06 15:03:16 on dist-test-slave-tc2s"
server_key: "a069f44b7ae2222d6392681e62d3774f"
server_key_iv: "864c63f7292b78a1d4e897994df8d159"
server_key_version: "encryptionkey@0"
I20260806 15:03:16.371510 5180 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:16.372234 5207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:16.372393 5180 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:16.372478 5180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "e5b91fff821d431f99f1ced497469760"
format_stamp: "Formatted at 2026-08-06 15:03:16 on dist-test-slave-tc2s"
server_key: "a069f44b7ae2222d6392681e62d3774f"
server_key_iv: "864c63f7292b78a1d4e897994df8d159"
server_key_version: "encryptionkey@0"
I20260806 15:03:16.372546 5180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:16.394522 5180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:16.396130 5180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:16.396247 5180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:16.399947 5180 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:45911
I20260806 15:03:16.399989 5283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:45911 every 8 connection(s)
I20260806 15:03:16.400275 5180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:03:16.401077 5285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:16.403249 5285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5b91fff821d431f99f1ced497469760: Bootstrap starting.
I20260806 15:03:16.403885 5285 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5b91fff821d431f99f1ced497469760: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:16.404150 5285 log.cc:824] T 00000000000000000000000000000000 P e5b91fff821d431f99f1ced497469760: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:16.404750 5285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5b91fff821d431f99f1ced497469760: No bootstrap required, opened a new log
I20260806 15:03:16.406332 5285 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5b91fff821d431f99f1ced497469760 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b91fff821d431f99f1ced497469760" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 45911 } }
I20260806 15:03:16.406462 5285 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5b91fff821d431f99f1ced497469760 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:16.406494 5285 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5b91fff821d431f99f1ced497469760 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5b91fff821d431f99f1ced497469760, State: Initialized, Role: FOLLOWER
I20260806 15:03:16.406618 5285 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5b91fff821d431f99f1ced497469760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b91fff821d431f99f1ced497469760" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 45911 } }
I20260806 15:03:16.406675 5285 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5b91fff821d431f99f1ced497469760 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:03:16.406705 5285 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5b91fff821d431f99f1ced497469760 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:03:16.406752 5285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5b91fff821d431f99f1ced497469760 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:16.407490 5285 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5b91fff821d431f99f1ced497469760 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b91fff821d431f99f1ced497469760" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 45911 } }
I20260806 15:03:16.407582 5285 leader_election.cc:302] T 00000000000000000000000000000000 P e5b91fff821d431f99f1ced497469760 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5b91fff821d431f99f1ced497469760; no voters:
I20260806 15:03:16.407740 5285 leader_election.cc:288] T 00000000000000000000000000000000 P e5b91fff821d431f99f1ced497469760 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:03:16.407779 5291 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5b91fff821d431f99f1ced497469760 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:16.407915 5291 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5b91fff821d431f99f1ced497469760 [term 1 LEADER]: Becoming Leader. State: Replica: e5b91fff821d431f99f1ced497469760, State: Running, Role: LEADER
I20260806 15:03:16.407976 5285 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5b91fff821d431f99f1ced497469760 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:03:16.408006 5291 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5b91fff821d431f99f1ced497469760 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b91fff821d431f99f1ced497469760" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 45911 } }
I20260806 15:03:16.408344 5293 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5b91fff821d431f99f1ced497469760 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5b91fff821d431f99f1ced497469760. Latest consensus state: current_term: 1 leader_uuid: "e5b91fff821d431f99f1ced497469760" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b91fff821d431f99f1ced497469760" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 45911 } } }
I20260806 15:03:16.408455 5293 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5b91fff821d431f99f1ced497469760 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:16.408339 5292 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5b91fff821d431f99f1ced497469760 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5b91fff821d431f99f1ced497469760" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b91fff821d431f99f1ced497469760" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 45911 } } }
I20260806 15:03:16.408635 5292 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5b91fff821d431f99f1ced497469760 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:16.408958 5301 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:03:16.409446 5301 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:03:16.410071 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 5180
I20260806 15:03:16.410168 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:03:16.410336 945 external_mini_cluster.cc:1428] Setting key 8a43de6150c8080749b8423448f95d65
I20260806 15:03:16.411129 5301 catalog_manager.cc:1347] Generated new cluster ID: 49ea8cd678f34e74aa624a3ae6c484ed
I20260806 15:03:16.411175 5301 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 15:03:16 dist-test-slave-tc2s krb5kdc[5164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:16.420692 5301 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:03:16.421404 5301 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:03:16.421672 5320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:16.410729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44314 (local address 127.0.236.126:45911)
0806 15:03:16.411183 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:16.411191 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:16.411224 (+ 33us) server_negotiation.cc:407] Connection header received
0806 15:03:16.411327 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:16.411333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:16.411386 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:16.411518 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:16.412206 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:16.413608 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:16.414614 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:16.415264 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:16.415378 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:16.418007 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:16.418011 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:16.418012 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:16.418023 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:16.419833 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:16.420236 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:16.420238 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:16.420239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:16.420534 (+ 295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:16.420737 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:16.420738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:16.420739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:16.420989 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:16.421131 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:16.421225 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 15:03:16.421477 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":416,"thread_start_us":75,"threads_started":1}
I20260806 15:03:16.422533 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:03:16.425805 5301 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5b91fff821d431f99f1ced497469760: Generated new TSK 0
I20260806 15:03:16.425981 5301 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:03:16.438236 5320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:16.434753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44324 (local address 127.0.236.126:45911)
0806 15:03:16.434804 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:16.434806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:16.434899 (+ 93us) server_negotiation.cc:407] Connection header received
0806 15:03:16.434962 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:16.434962 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:16.434972 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:16.435025 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:16.435339 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:16.436106 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:16.436791 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:16.437129 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:16.437194 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:16.437772 (+ 578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:16.437773 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:16.437774 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:03:16.438010 (+ 236us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:16.438028 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:03:16.438036 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:16.438089 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:16.438134 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 15:03:16.438160 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 15:03:16.440699 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 5180
2026-08-06T15:03:16Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1399 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:03:16 dist-test-slave-tc2s krb5kdc[5333](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:03:16 dist-test-slave-tc2s krb5kdc[5333](info): set up 2 sockets
Aug 06 15:03:16 dist-test-slave-tc2s krb5kdc[5333](info): commencing operation
krb5kdc: starting...
W20260806 15:03:17.549320 945 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:03:17 dist-test-slave-tc2s krb5kdc[5333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:03:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:03:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:03:17.657213 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:36441
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:46669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:36441
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:17.735373 5351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:17.735553 5351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:17.735584 5351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:17.736658 5351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:03:17.736696 5351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:17.736713 5351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:17.736728 5351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:03:17.736744 5351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:03:17.738226 5351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:46669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:36441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:36441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:17.738453 5351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:17.738654 5351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:17.743074 5356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:17.743080 5357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:17.743072 5360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:17.743472 5351 server_base.cc:1034] running on GCE node
I20260806 15:03:17.743690 5351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:17.743916 5351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:17.745051 5351 hybrid_clock.cc:648] HybridClock initialized: now 1786028597745038 us; error 29 us; skew 500 ppm
Aug 06 15:03:17 dist-test-slave-tc2s krb5kdc[5333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028597, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:17.747346 5351 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:03:17.747738 5351 webserver.cc:460] Webserver started at http://127.0.236.126:36403/ using document root <none> and password file <none>
I20260806 15:03:17.747931 5351 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:17.747969 5351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:17.748057 5351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:17.748984 5351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "391dc9d5eb7e4fa2b27a724f8bc76b1b"
format_stamp: "Formatted at 2026-08-06 15:03:17 on dist-test-slave-tc2s"
server_key: "20753d1e287967be033c581c7123a9e8"
server_key_iv: "f1f9b2292d9ed91e489edf16ba01c525"
server_key_version: "encryptionkey@0"
I20260806 15:03:17.749267 5351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "391dc9d5eb7e4fa2b27a724f8bc76b1b"
format_stamp: "Formatted at 2026-08-06 15:03:17 on dist-test-slave-tc2s"
server_key: "20753d1e287967be033c581c7123a9e8"
server_key_iv: "f1f9b2292d9ed91e489edf16ba01c525"
server_key_version: "encryptionkey@0"
I20260806 15:03:17.750491 5351 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 15:03:17.751132 5368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:17.751268 5351 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:17.751344 5351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "391dc9d5eb7e4fa2b27a724f8bc76b1b"
format_stamp: "Formatted at 2026-08-06 15:03:17 on dist-test-slave-tc2s"
server_key: "20753d1e287967be033c581c7123a9e8"
server_key_iv: "f1f9b2292d9ed91e489edf16ba01c525"
server_key_version: "encryptionkey@0"
I20260806 15:03:17.751406 5351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:17.771736 5351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:17.773257 5351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:17.773375 5351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:17.777093 5351 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:36441
I20260806 15:03:17.777124 5451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:36441 every 8 connection(s)
I20260806 15:03:17.777407 5351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:03:17.778512 5452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:17.780591 5452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 391dc9d5eb7e4fa2b27a724f8bc76b1b: Bootstrap starting.
I20260806 15:03:17.781131 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 5351
I20260806 15:03:17.781188 5452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 391dc9d5eb7e4fa2b27a724f8bc76b1b: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:17.781241 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:03:17.781410 945 external_mini_cluster.cc:1428] Setting key 0a5f173402534d94291672365b0983c2
I20260806 15:03:17.781493 5452 log.cc:824] T 00000000000000000000000000000000 P 391dc9d5eb7e4fa2b27a724f8bc76b1b: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:17.782486 5452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 391dc9d5eb7e4fa2b27a724f8bc76b1b: No bootstrap required, opened a new log
I20260806 15:03:17.784538 5452 raft_consensus.cc:348] T 00000000000000000000000000000000 P 391dc9d5eb7e4fa2b27a724f8bc76b1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "391dc9d5eb7e4fa2b27a724f8bc76b1b" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 36441 } }
I20260806 15:03:17.784699 5452 raft_consensus.cc:374] T 00000000000000000000000000000000 P 391dc9d5eb7e4fa2b27a724f8bc76b1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:17.784734 5452 raft_consensus.cc:729] T 00000000000000000000000000000000 P 391dc9d5eb7e4fa2b27a724f8bc76b1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 391dc9d5eb7e4fa2b27a724f8bc76b1b, State: Initialized, Role: FOLLOWER
I20260806 15:03:17.784848 5452 consensus_queue.cc:260] T 00000000000000000000000000000000 P 391dc9d5eb7e4fa2b27a724f8bc76b1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "391dc9d5eb7e4fa2b27a724f8bc76b1b" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 36441 } }
I20260806 15:03:17.784937 5452 raft_consensus.cc:388] T 00000000000000000000000000000000 P 391dc9d5eb7e4fa2b27a724f8bc76b1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:03:17.784970 5452 raft_consensus.cc:482] T 00000000000000000000000000000000 P 391dc9d5eb7e4fa2b27a724f8bc76b1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:03:17.785012 5452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 391dc9d5eb7e4fa2b27a724f8bc76b1b [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:17.785571 5452 raft_consensus.cc:504] T 00000000000000000000000000000000 P 391dc9d5eb7e4fa2b27a724f8bc76b1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "391dc9d5eb7e4fa2b27a724f8bc76b1b" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 36441 } }
I20260806 15:03:17.785658 5452 leader_election.cc:302] T 00000000000000000000000000000000 P 391dc9d5eb7e4fa2b27a724f8bc76b1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 391dc9d5eb7e4fa2b27a724f8bc76b1b; no voters:
I20260806 15:03:17.785816 5452 leader_election.cc:288] T 00000000000000000000000000000000 P 391dc9d5eb7e4fa2b27a724f8bc76b1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:03:17.785887 5457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 391dc9d5eb7e4fa2b27a724f8bc76b1b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:17.786092 5457 raft_consensus.cc:686] T 00000000000000000000000000000000 P 391dc9d5eb7e4fa2b27a724f8bc76b1b [term 1 LEADER]: Becoming Leader. State: Replica: 391dc9d5eb7e4fa2b27a724f8bc76b1b, State: Running, Role: LEADER
I20260806 15:03:17.786099 5452 sys_catalog.cc:564] T 00000000000000000000000000000000 P 391dc9d5eb7e4fa2b27a724f8bc76b1b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:03:17.786196 5457 consensus_queue.cc:237] T 00000000000000000000000000000000 P 391dc9d5eb7e4fa2b27a724f8bc76b1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "391dc9d5eb7e4fa2b27a724f8bc76b1b" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 36441 } }
I20260806 15:03:17.786504 5458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 391dc9d5eb7e4fa2b27a724f8bc76b1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "391dc9d5eb7e4fa2b27a724f8bc76b1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "391dc9d5eb7e4fa2b27a724f8bc76b1b" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 36441 } } }
I20260806 15:03:17.786510 5460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 391dc9d5eb7e4fa2b27a724f8bc76b1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 391dc9d5eb7e4fa2b27a724f8bc76b1b. Latest consensus state: current_term: 1 leader_uuid: "391dc9d5eb7e4fa2b27a724f8bc76b1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "391dc9d5eb7e4fa2b27a724f8bc76b1b" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 36441 } } }
I20260806 15:03:17.786607 5458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 391dc9d5eb7e4fa2b27a724f8bc76b1b [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:17.786608 5460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 391dc9d5eb7e4fa2b27a724f8bc76b1b [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:17.786821 5466 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:03:17.787289 5466 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 15:03:17 dist-test-slave-tc2s krb5kdc[5333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:17.788807 5466 catalog_manager.cc:1347] Generated new cluster ID: 0dcda42f80b74a8ebc1d3a695896a081
I20260806 15:03:17.788856 5466 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:03:17.792076 5455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:17.781871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59074 (local address 127.0.236.126:36441)
0806 15:03:17.782152 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:17.782162 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:17.782194 (+ 32us) server_negotiation.cc:407] Connection header received
0806 15:03:17.782622 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:17.782629 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:17.782675 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:17.782784 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:17.783493 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:17.784302 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:17.785311 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:17.785747 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:17.785849 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:17.788695 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:17.788699 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:17.788700 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:17.788710 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:17.790569 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:17.790926 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:17.790929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:17.790929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:17.791039 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:17.791205 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:17.791206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:17.791206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:17.791511 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:17.791665 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:17.791745 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 15:03:17.791877 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":83,"threads_started":1}
I20260806 15:03:17.807792 5466 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:03:17.808470 5466 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:03:17.819620 5466 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 391dc9d5eb7e4fa2b27a724f8bc76b1b: Generated new TSK 0
I20260806 15:03:17.819772 5466 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.236.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
I20260806 15:03:17.848534 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:0
--local_ip_for_outbound_sockets=127.0.236.65
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:36441
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:46669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:17.926832 5487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:17.927013 5487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:17.927044 5487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:17.928120 5487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:17.928158 5487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:17.928191 5487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:03:17.929585 5487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:46669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:36441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:17.929796 5487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:17.930019 5487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:17.934471 5501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:17.934499 5504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:17.934653 5487 server_base.cc:1034] running on GCE node
W20260806 15:03:17.934510 5499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:17.934890 5487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:17.935111 5487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:17.936257 5487 hybrid_clock.cc:648] HybridClock initialized: now 1786028597936246 us; error 31 us; skew 500 ppm
Aug 06 15:03:17 dist-test-slave-tc2s krb5kdc[5333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028597, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:17.938628 5487 init.cc:377] Logged in from keytab as kudu/127.0.236.65@KRBTEST.COM (short username kudu)
I20260806 15:03:17.939020 5487 webserver.cc:460] Webserver started at http://127.0.236.65:42695/ using document root <none> and password file <none>
I20260806 15:03:17.939209 5487 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:17.939247 5487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:17.939332 5487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:17.940217 5487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/data/instance:
uuid: "125468dccb7f4217ac4cab75160b12b7"
format_stamp: "Formatted at 2026-08-06 15:03:17 on dist-test-slave-tc2s"
server_key: "52fb0fb5fb8a22e1501d3eb11027aa43"
server_key_iv: "ea33ca04123aed056056e3b6cb3cf016"
server_key_version: "encryptionkey@0"
I20260806 15:03:17.940507 5487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/wal/instance:
uuid: "125468dccb7f4217ac4cab75160b12b7"
format_stamp: "Formatted at 2026-08-06 15:03:17 on dist-test-slave-tc2s"
server_key: "52fb0fb5fb8a22e1501d3eb11027aa43"
server_key_iv: "ea33ca04123aed056056e3b6cb3cf016"
server_key_version: "encryptionkey@0"
I20260806 15:03:17.941603 5487 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 15:03:17.942237 5512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:17.942379 5487 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:17.942463 5487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "125468dccb7f4217ac4cab75160b12b7"
format_stamp: "Formatted at 2026-08-06 15:03:17 on dist-test-slave-tc2s"
server_key: "52fb0fb5fb8a22e1501d3eb11027aa43"
server_key_iv: "ea33ca04123aed056056e3b6cb3cf016"
server_key_version: "encryptionkey@0"
I20260806 15:03:17.942521 5487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:17.955186 5487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:17.957057 5487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:17.957177 5487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:17.957360 5487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:17.957625 5487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:17.957657 5487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:17.957682 5487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:17.957707 5487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:17.963008 5487 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:42173
I20260806 15:03:17.963061 5682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:42173 every 8 connection(s)
I20260806 15:03:17.963307 5487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
Aug 06 15:03:17 dist-test-slave-tc2s krb5kdc[5333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028597, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:17.971217 5455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:17.963895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:36647 (local address 127.0.236.126:36441)
0806 15:03:17.963940 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:17.963942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:17.964158 (+ 216us) server_negotiation.cc:407] Connection header received
0806 15:03:17.964698 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:17.964700 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:17.964714 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:17.964814 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:17.965679 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:17.966228 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:17.967121 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:17.967307 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:17.967349 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:17.969157 (+ 1808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:17.969158 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:17.969158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:17.969164 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:17.970235 (+ 1071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:17.970566 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:17.970566 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:17.970567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:17.970612 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:17.970773 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:17.970773 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:17.970773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:17.970863 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:17.970906 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:17.971093 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 15:03:17.971156 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 15:03:17.971339 5686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:17.963887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:36441 (local address 127.0.236.65:36647)
0806 15:03:17.964036 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:17.964068 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:17.964576 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:17.964876 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:17.964880 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:17.965434 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:17.965604 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:17.965608 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:17.966291 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:17.966292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:17.967060 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:17.967063 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:17.967358 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:17.967359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:17.967456 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:17.967854 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:17.967862 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:17.969087 (+ 1225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:17.970305 (+ 1218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:17.970311 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:17.970312 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:17.970516 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:17.970653 (+ 137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:17.970653 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:17.970654 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:17.970734 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:17.970902 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:17.970905 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:17.971006 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 15:03:17.971076 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 15:03:17.971180 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":92,"threads_started":1}
I20260806 15:03:17.971789 5683 heartbeater.cc:347] Connected to a master server at 127.0.236.126:36441
I20260806 15:03:17.971870 5683 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:17.972033 5683 heartbeater.cc:511] Master 127.0.236.126:36441 requested a full tablet report, sending...
I20260806 15:03:17.972500 5398 ts_manager.cc:194] Registered new tserver with Master: 125468dccb7f4217ac4cab75160b12b7 (127.0.236.65:42173)
I20260806 15:03:17.973074 5398 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.65@KRBTEST.COM'} at 127.0.236.65:36647
I20260806 15:03:17.973207 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 5487
I20260806 15:03:17.973302 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-0/wal/instance
I20260806 15:03:17.973505 945 external_mini_cluster.cc:1428] Setting key 78d1259fd1a008cb7a37149b3a0d8069
WARNING: no policy specified for kudu/127.0.236.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
I20260806 15:03:17.994685 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:0
--local_ip_for_outbound_sockets=127.0.236.66
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:36441
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:46669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:18.071945 5691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:18.072125 5691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:18.072155 5691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:18.073239 5691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:18.073279 5691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:18.073312 5691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:03:18.074765 5691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:46669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:36441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:18.074990 5691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:18.075197 5691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:18.079810 5702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:18.079849 5698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:18.079816 5700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:18.080017 5691 server_base.cc:1034] running on GCE node
I20260806 15:03:18.080219 5691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:18.080461 5691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:18.081597 5691 hybrid_clock.cc:648] HybridClock initialized: now 1786028598081585 us; error 24 us; skew 500 ppm
Aug 06 15:03:18 dist-test-slave-tc2s krb5kdc[5333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028598, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:18.083841 5691 init.cc:377] Logged in from keytab as kudu/127.0.236.66@KRBTEST.COM (short username kudu)
I20260806 15:03:18.084240 5691 webserver.cc:460] Webserver started at http://127.0.236.66:45557/ using document root <none> and password file <none>
I20260806 15:03:18.084458 5691 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:18.084506 5691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:18.084625 5691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:18.085564 5691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/data/instance:
uuid: "365ab215781a4145910aae228dfde24d"
format_stamp: "Formatted at 2026-08-06 15:03:18 on dist-test-slave-tc2s"
server_key: "2b715929e97e7b068b0790ecd2909b99"
server_key_iv: "97c612faf83ae88f86ba4f471b8bc4b9"
server_key_version: "encryptionkey@0"
I20260806 15:03:18.085868 5691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/wal/instance:
uuid: "365ab215781a4145910aae228dfde24d"
format_stamp: "Formatted at 2026-08-06 15:03:18 on dist-test-slave-tc2s"
server_key: "2b715929e97e7b068b0790ecd2909b99"
server_key_iv: "97c612faf83ae88f86ba4f471b8bc4b9"
server_key_version: "encryptionkey@0"
I20260806 15:03:18.087160 5691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:18.087801 5711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:18.087967 5691 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:18.088050 5691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "365ab215781a4145910aae228dfde24d"
format_stamp: "Formatted at 2026-08-06 15:03:18 on dist-test-slave-tc2s"
server_key: "2b715929e97e7b068b0790ecd2909b99"
server_key_iv: "97c612faf83ae88f86ba4f471b8bc4b9"
server_key_version: "encryptionkey@0"
I20260806 15:03:18.088114 5691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:18.117056 5691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:18.118690 5691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:18.118820 5691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:18.119020 5691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:18.119304 5691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:18.119338 5691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:18.119365 5691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:18.119382 5691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:18.124610 5691 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:41307
I20260806 15:03:18.124646 5883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:41307 every 8 connection(s)
I20260806 15:03:18.124907 5691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
I20260806 15:03:18.128615 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 5691
I20260806 15:03:18.128713 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-1/wal/instance
I20260806 15:03:18.128877 945 external_mini_cluster.cc:1428] Setting key 015b7303c354512ca12dbac6f8bab1b3
Aug 06 15:03:18 dist-test-slave-tc2s krb5kdc[5333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028598, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:18.133246 5455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:18.125563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:51171 (local address 127.0.236.126:36441)
0806 15:03:18.125611 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:18.125613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:18.125779 (+ 166us) server_negotiation.cc:407] Connection header received
0806 15:03:18.126228 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:18.126229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:18.126239 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:18.126287 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:18.127104 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:18.127582 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:18.128458 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:18.128626 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:18.128678 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:18.130670 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:18.130672 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:18.130672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:18.130680 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:18.132037 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:18.132389 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:18.132391 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:18.132391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:18.132450 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:18.132657 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:18.132658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:18.132658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:18.132770 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:18.132826 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:18.133074 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 15:03:18.133157 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:03:18.133383 5887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:18.125532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:36441 (local address 127.0.236.66:51171)
0806 15:03:18.125677 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:18.125708 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:18.126154 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:18.126341 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:18.126344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:18.126858 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:18.127039 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:18.127044 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:18.127640 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:18.127640 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:18.128402 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:18.128403 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:18.128711 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:18.128712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:18.128840 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:18.129289 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:18.129297 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:18.130588 (+ 1291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:18.132113 (+ 1525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:18.132119 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:18.132121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:18.132322 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:18.132503 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:18.132504 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:18.132504 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:18.132600 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:18.132824 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:18.132830 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:18.132955 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 15:03:18.133041 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 15:03:18.133184 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
I20260806 15:03:18.133813 5884 heartbeater.cc:347] Connected to a master server at 127.0.236.126:36441
I20260806 15:03:18.133937 5884 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:18.134074 5884 heartbeater.cc:511] Master 127.0.236.126:36441 requested a full tablet report, sending...
I20260806 15:03:18.134375 5398 ts_manager.cc:194] Registered new tserver with Master: 365ab215781a4145910aae228dfde24d (127.0.236.66:41307)
I20260806 15:03:18.134735 5398 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.66@KRBTEST.COM'} at 127.0.236.66:51171
WARNING: no policy specified for kudu/127.0.236.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
I20260806 15:03:18.151882 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:0
--local_ip_for_outbound_sockets=127.0.236.67
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:36441
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:46669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:18.229002 5894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:18.229171 5894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:18.229192 5894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:18.230245 5894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:18.230276 5894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:18.230300 5894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:03:18.231642 5894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:46669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:36441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:18.231810 5894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:18.231973 5894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:18.236500 5900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:18.236554 5894 server_base.cc:1034] running on GCE node
W20260806 15:03:18.236572 5902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:18.236500 5904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:18.236836 5894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:18.237025 5894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:18.238147 5894 hybrid_clock.cc:648] HybridClock initialized: now 1786028598238136 us; error 28 us; skew 500 ppm
Aug 06 15:03:18 dist-test-slave-tc2s krb5kdc[5333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028598, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:18.244223 5894 init.cc:377] Logged in from keytab as kudu/127.0.236.67@KRBTEST.COM (short username kudu)
I20260806 15:03:18.244618 5894 webserver.cc:460] Webserver started at http://127.0.236.67:44893/ using document root <none> and password file <none>
I20260806 15:03:18.244801 5894 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:18.244839 5894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:18.244926 5894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:18.245877 5894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/data/instance:
uuid: "9db9d4178beb412ba2ac752ad22ce07b"
format_stamp: "Formatted at 2026-08-06 15:03:18 on dist-test-slave-tc2s"
server_key: "a80f8cb439d63b1cce739c6d5a48d731"
server_key_iv: "bf7d6e0749997c700ce4f2545aacafdf"
server_key_version: "encryptionkey@0"
I20260806 15:03:18.246222 5894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/wal/instance:
uuid: "9db9d4178beb412ba2ac752ad22ce07b"
format_stamp: "Formatted at 2026-08-06 15:03:18 on dist-test-slave-tc2s"
server_key: "a80f8cb439d63b1cce739c6d5a48d731"
server_key_iv: "bf7d6e0749997c700ce4f2545aacafdf"
server_key_version: "encryptionkey@0"
I20260806 15:03:18.247324 5894 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:03:18.247924 5919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:18.248070 5894 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:03:18.248138 5894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "9db9d4178beb412ba2ac752ad22ce07b"
format_stamp: "Formatted at 2026-08-06 15:03:18 on dist-test-slave-tc2s"
server_key: "a80f8cb439d63b1cce739c6d5a48d731"
server_key_iv: "bf7d6e0749997c700ce4f2545aacafdf"
server_key_version: "encryptionkey@0"
I20260806 15:03:18.248201 5894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:18.286425 5894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:18.288445 5894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:18.288566 5894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:18.288753 5894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:18.289017 5894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:18.289048 5894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:18.289068 5894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:18.289090 5894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:18.294593 5894 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:34691
I20260806 15:03:18.294632 6087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:34691 every 8 connection(s)
I20260806 15:03:18.294888 5894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
I20260806 15:03:18.296156 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 5894
I20260806 15:03:18.296250 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028580791223-945-0/minicluster-data/ts-2/wal/instance
I20260806 15:03:18.296423 945 external_mini_cluster.cc:1428] Setting key 8225a69e13fc1136e459b6477062fd1b
Aug 06 15:03:18 dist-test-slave-tc2s krb5kdc[5333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028598, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:18.302852 5455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:18.295478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:52153 (local address 127.0.236.126:36441)
0806 15:03:18.295518 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:18.295520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:18.295683 (+ 163us) server_negotiation.cc:407] Connection header received
0806 15:03:18.296163 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:18.296163 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:18.296173 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:18.296214 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:18.297291 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:18.297783 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:18.298718 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:18.298900 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:18.298944 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:18.300749 (+ 1805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:18.300751 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:18.300751 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:18.300758 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:18.301838 (+ 1080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:18.302190 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:18.302190 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:18.302191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:18.302239 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:18.302404 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:18.302405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:18.302405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:18.302505 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:18.302550 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:18.302736 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 15:03:18.302791 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 15:03:18.302995 6093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:18.295457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:36441 (local address 127.0.236.67:52153)
0806 15:03:18.295591 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:18.295621 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:18.296068 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:18.296281 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:18.296285 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:18.296947 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:18.297202 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:18.297208 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:18.297842 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:18.297843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:18.298650 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:18.298653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:18.298957 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:18.298958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:18.299048 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:18.299399 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:18.299407 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:18.300671 (+ 1264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:18.301925 (+ 1254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:18.301931 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:18.301933 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:18.302129 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:18.302282 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:18.302282 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:18.302283 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:18.302365 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:18.302547 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:18.302550 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:18.302648 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 15:03:18.302719 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 15:03:18.302826 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":78,"threads_started":1}
I20260806 15:03:18.303383 6090 heartbeater.cc:347] Connected to a master server at 127.0.236.126:36441
I20260806 15:03:18.303479 6090 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:18.303628 6090 heartbeater.cc:511] Master 127.0.236.126:36441 requested a full tablet report, sending...
I20260806 15:03:18.303944 5398 ts_manager.cc:194] Registered new tserver with Master: 9db9d4178beb412ba2ac752ad22ce07b (127.0.236.67:34691)
I20260806 15:03:18.304267 5398 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.67@KRBTEST.COM'} at 127.0.236.67:52153
I20260806 15:03:18.307832 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 15:03:18 dist-test-slave-tc2s krb5kdc[5333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028598, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 15:03:18 dist-test-slave-tc2s krb5kdc[5333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028598, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:18.329361 5455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:18.323428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59076 (local address 127.0.236.126:36441)
0806 15:03:18.323482 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:18.323485 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:18.323585 (+ 100us) server_negotiation.cc:407] Connection header received
0806 15:03:18.323647 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:18.323648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:18.323657 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:18.323701 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:18.324214 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:18.324682 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:18.325378 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:18.325566 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:18.325620 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:18.327556 (+ 1936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:18.327557 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:18.327557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:18.327563 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:18.328552 (+ 989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:18.328824 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:18.328825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:18.328825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:18.328865 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:18.328986 (+ 121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:18.328987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:18.328987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:18.329106 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:18.329179 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:18.329245 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 15:03:18.329302 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260806 15:03:18.329552 5398 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59076
I20260806 15:03:18.330034 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 5487
I20260806 15:03:18.352422 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 5691
I20260806 15:03:18.361368 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 5894
I20260806 15:03:18.368451 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 5351
2026-08-06T15:03:18Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1921 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:03:18 dist-test-slave-tc2s krb5kdc[6110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:03:18 dist-test-slave-tc2s krb5kdc[6110](info): set up 2 sockets
Aug 06 15:03:18 dist-test-slave-tc2s krb5kdc[6110](info): commencing operation
krb5kdc: starting...
W20260806 15:03:19.473110 945 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:03:19 dist-test-slave-tc2s krb5kdc[6110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:03:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:03:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:03:19.581413 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:44569
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:34141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:44569
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:19.661144 6126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:19.661342 6126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:19.661373 6126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:19.662509 6126 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:03:19.662547 6126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:19.662566 6126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:19.662581 6126 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:03:19.662597 6126 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:03:19.663997 6126 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:34141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:44569
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:44569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:19.664229 6126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:19.664407 6126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:19.668810 6136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:19.668828 6138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:19.668937 6126 server_base.cc:1034] running on GCE node
W20260806 15:03:19.668818 6135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:19.669258 6126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:19.669534 6126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:19.670667 6126 hybrid_clock.cc:648] HybridClock initialized: now 1786028599670653 us; error 27 us; skew 500 ppm
Aug 06 15:03:19 dist-test-slave-tc2s krb5kdc[6110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028599, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:19.672923 6126 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:03:19.673323 6126 webserver.cc:460] Webserver started at http://127.0.236.126:43325/ using document root <none> and password file <none>
I20260806 15:03:19.673514 6126 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:19.673559 6126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:19.673678 6126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:19.674655 6126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "e8930b9b137947fc849b354229091283"
format_stamp: "Formatted at 2026-08-06 15:03:19 on dist-test-slave-tc2s"
server_key: "299c4bfd54ff08f369d9ce7dc5818a34"
server_key_iv: "47fe400c79ef7cfc8f0fe604f440c30c"
server_key_version: "encryptionkey@0"
I20260806 15:03:19.674969 6126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "e8930b9b137947fc849b354229091283"
format_stamp: "Formatted at 2026-08-06 15:03:19 on dist-test-slave-tc2s"
server_key: "299c4bfd54ff08f369d9ce7dc5818a34"
server_key_iv: "47fe400c79ef7cfc8f0fe604f440c30c"
server_key_version: "encryptionkey@0"
I20260806 15:03:19.676146 6126 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 15:03:19.676756 6150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:19.676901 6126 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:03:19.676982 6126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "e8930b9b137947fc849b354229091283"
format_stamp: "Formatted at 2026-08-06 15:03:19 on dist-test-slave-tc2s"
server_key: "299c4bfd54ff08f369d9ce7dc5818a34"
server_key_iv: "47fe400c79ef7cfc8f0fe604f440c30c"
server_key_version: "encryptionkey@0"
I20260806 15:03:19.677044 6126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:19.685261 6126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:19.687180 6126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:19.687304 6126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:19.691079 6126 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:44569
I20260806 15:03:19.691118 6226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:44569 every 8 connection(s)
I20260806 15:03:19.691403 6126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:03:19.692147 6229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:19.694243 6229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8930b9b137947fc849b354229091283: Bootstrap starting.
I20260806 15:03:19.694748 6229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8930b9b137947fc849b354229091283: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:19.694988 6229 log.cc:824] T 00000000000000000000000000000000 P e8930b9b137947fc849b354229091283: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:19.695605 6229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8930b9b137947fc849b354229091283: No bootstrap required, opened a new log
I20260806 15:03:19.695663 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 6126
I20260806 15:03:19.695763 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:03:19.695961 945 external_mini_cluster.cc:1428] Setting key 03b661d77ed522d943f3e457efaba01e
I20260806 15:03:19.697409 6229 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8930b9b137947fc849b354229091283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8930b9b137947fc849b354229091283" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44569 } }
I20260806 15:03:19.697557 6229 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8930b9b137947fc849b354229091283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:19.697635 6229 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8930b9b137947fc849b354229091283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8930b9b137947fc849b354229091283, State: Initialized, Role: FOLLOWER
I20260806 15:03:19.697767 6229 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8930b9b137947fc849b354229091283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8930b9b137947fc849b354229091283" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44569 } }
I20260806 15:03:19.697846 6229 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8930b9b137947fc849b354229091283 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:03:19.697885 6229 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8930b9b137947fc849b354229091283 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:03:19.697963 6229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8930b9b137947fc849b354229091283 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:19.698920 6229 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8930b9b137947fc849b354229091283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8930b9b137947fc849b354229091283" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44569 } }
I20260806 15:03:19.699007 6229 leader_election.cc:302] T 00000000000000000000000000000000 P e8930b9b137947fc849b354229091283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8930b9b137947fc849b354229091283; no voters:
I20260806 15:03:19.699150 6229 leader_election.cc:288] T 00000000000000000000000000000000 P e8930b9b137947fc849b354229091283 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:03:19.699189 6234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8930b9b137947fc849b354229091283 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:19.699326 6234 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8930b9b137947fc849b354229091283 [term 1 LEADER]: Becoming Leader. State: Replica: e8930b9b137947fc849b354229091283, State: Running, Role: LEADER
I20260806 15:03:19.699404 6229 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8930b9b137947fc849b354229091283 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:03:19.699411 6234 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8930b9b137947fc849b354229091283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8930b9b137947fc849b354229091283" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44569 } }
I20260806 15:03:19.700024 6239 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8930b9b137947fc849b354229091283 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8930b9b137947fc849b354229091283" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8930b9b137947fc849b354229091283" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44569 } } }
I20260806 15:03:19.700109 6239 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8930b9b137947fc849b354229091283 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:19.700030 6241 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8930b9b137947fc849b354229091283 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8930b9b137947fc849b354229091283. Latest consensus state: current_term: 1 leader_uuid: "e8930b9b137947fc849b354229091283" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8930b9b137947fc849b354229091283" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44569 } } }
I20260806 15:03:19.700150 6241 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8930b9b137947fc849b354229091283 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:19.700404 6248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:03:19.700902 6248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:03:19.702764 6248 catalog_manager.cc:1347] Generated new cluster ID: 1b26c3bc4ba24e00897a9d36e2ac66b3
I20260806 15:03:19.702813 6248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 15:03:19 dist-test-slave-tc2s krb5kdc[6110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:19.707527 6233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:19.696423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59244 (local address 127.0.236.126:44569)
0806 15:03:19.696577 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:19.696583 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:19.696607 (+ 24us) server_negotiation.cc:407] Connection header received
0806 15:03:19.697378 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:19.697384 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:19.697434 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:19.697534 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:19.698305 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:19.699267 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:19.700248 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:19.700852 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:19.700952 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:19.703618 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:19.703622 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:19.703622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:19.703632 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:19.705366 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:19.705752 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:19.705754 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:19.705754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:19.706390 (+ 636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:19.706610 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:19.706610 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:19.706610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:19.706878 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:19.707037 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:19.707134 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 15:03:19.707255 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260806 15:03:19.710590 6248 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:03:19.711278 6248 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:03:19.715014 6248 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8930b9b137947fc849b354229091283: Generated new TSK 0
I20260806 15:03:19.715162 6248 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.236.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
I20260806 15:03:19.741776 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:0
--local_ip_for_outbound_sockets=127.0.236.65
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44569
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:34141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:19.820153 6271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:19.820353 6271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:19.820384 6271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:19.821501 6271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:19.821538 6271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:19.821571 6271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:03:19.823019 6271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:34141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:19.823240 6271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:19.823443 6271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:19.827958 6278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:19.827991 6285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:19.828001 6276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:19.828202 6271 server_base.cc:1034] running on GCE node
I20260806 15:03:19.828405 6271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:19.828619 6271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:19.829768 6271 hybrid_clock.cc:648] HybridClock initialized: now 1786028599829750 us; error 32 us; skew 500 ppm
Aug 06 15:03:19 dist-test-slave-tc2s krb5kdc[6110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028599, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:19.832118 6271 init.cc:377] Logged in from keytab as kudu/127.0.236.65@KRBTEST.COM (short username kudu)
I20260806 15:03:19.832497 6271 webserver.cc:460] Webserver started at http://127.0.236.65:35423/ using document root <none> and password file <none>
I20260806 15:03:19.832703 6271 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:19.832752 6271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:19.832875 6271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:19.833807 6271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-0/data/instance:
uuid: "64e9d4571d814580b7848dcf6f096097"
format_stamp: "Formatted at 2026-08-06 15:03:19 on dist-test-slave-tc2s"
server_key: "12dcb3f451f2843b90c5aed487657913"
server_key_iv: "bfac1b6cae7b07faf9801af5a7567820"
server_key_version: "encryptionkey@0"
I20260806 15:03:19.834177 6271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-0/wal/instance:
uuid: "64e9d4571d814580b7848dcf6f096097"
format_stamp: "Formatted at 2026-08-06 15:03:19 on dist-test-slave-tc2s"
server_key: "12dcb3f451f2843b90c5aed487657913"
server_key_iv: "bfac1b6cae7b07faf9801af5a7567820"
server_key_version: "encryptionkey@0"
I20260806 15:03:19.835419 6271 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:19.836082 6294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:19.836230 6271 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:19.836313 6271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "64e9d4571d814580b7848dcf6f096097"
format_stamp: "Formatted at 2026-08-06 15:03:19 on dist-test-slave-tc2s"
server_key: "12dcb3f451f2843b90c5aed487657913"
server_key_iv: "bfac1b6cae7b07faf9801af5a7567820"
server_key_version: "encryptionkey@0"
I20260806 15:03:19.836380 6271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:19.846858 6271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:19.848407 6271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:19.848527 6271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:19.848706 6271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:19.848970 6271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:19.849002 6271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:19.849028 6271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:19.849053 6271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:19.854435 6271 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:34409
I20260806 15:03:19.854492 6466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:34409 every 8 connection(s)
I20260806 15:03:19.854737 6271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:03:19.856329 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 6271
I20260806 15:03:19.856428 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-0/wal/instance
I20260806 15:03:19.856616 945 external_mini_cluster.cc:1428] Setting key 38f699de7bd8ae11baef84fead4f5339
Aug 06 15:03:19 dist-test-slave-tc2s krb5kdc[6110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028599, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:19.863899 6233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:19.855351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:60895 (local address 127.0.236.126:44569)
0806 15:03:19.855401 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:19.855403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:19.855576 (+ 173us) server_negotiation.cc:407] Connection header received
0806 15:03:19.856434 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:19.856435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:19.856448 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:19.856513 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:19.857163 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:19.857682 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:19.859049 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:19.859345 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:19.859412 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:19.861395 (+ 1983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:19.861397 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:19.861397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:19.861405 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:19.862667 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:19.863029 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:19.863031 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:19.863031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:19.863085 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:19.863297 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:19.863298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:19.863298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:19.863406 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:19.863463 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:19.863710 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 15:03:19.863796 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 15:03:19.864070 6477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:19.855303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:44569 (local address 127.0.236.65:60895)
0806 15:03:19.855466 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:19.855503 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:19.856314 (+ 811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:19.856590 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:19.856593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:19.856867 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:19.857074 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:19.857078 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:19.857776 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:19.857777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:19.858956 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:19.858959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:19.859413 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:19.859413 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:19.859511 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:19.859904 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:19.859912 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:19.861300 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:19.862740 (+ 1440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:19.862748 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:19.862750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:19.862962 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:19.863138 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:19.863139 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:19.863139 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:19.863242 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:19.863462 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:19.863469 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:19.863596 (+ 127us) client_negotiation.cc:769] Sending connection context
0806 15:03:19.863677 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 15:03:19.863820 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
I20260806 15:03:19.864521 6469 heartbeater.cc:347] Connected to a master server at 127.0.236.126:44569
I20260806 15:03:19.864619 6469 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:19.864804 6469 heartbeater.cc:511] Master 127.0.236.126:44569 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.0.236.66@KRBTEST.COM; defaulting to no policy
I20260806 15:03:19.865290 6178 ts_manager.cc:194] Registered new tserver with Master: 64e9d4571d814580b7848dcf6f096097 (127.0.236.65:34409)
I20260806 15:03:19.865912 6178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.65@KRBTEST.COM'} at 127.0.236.65:60895
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
I20260806 15:03:19.880518 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:0
--local_ip_for_outbound_sockets=127.0.236.66
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44569
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:34141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:19.957976 6482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:19.958153 6482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:19.958174 6482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:19.959262 6482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:19.959300 6482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:19.959336 6482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:03:19.960775 6482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:34141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:19.960980 6482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:19.961221 6482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:19.965941 6488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:19.966054 6482 server_base.cc:1034] running on GCE node
W20260806 15:03:19.965950 6490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:19.965950 6487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:19.966356 6482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:19.966572 6482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:19.967716 6482 hybrid_clock.cc:648] HybridClock initialized: now 1786028599967704 us; error 26 us; skew 500 ppm
Aug 06 15:03:19 dist-test-slave-tc2s krb5kdc[6110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028599, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:19.970031 6482 init.cc:377] Logged in from keytab as kudu/127.0.236.66@KRBTEST.COM (short username kudu)
I20260806 15:03:19.970417 6482 webserver.cc:460] Webserver started at http://127.0.236.66:34381/ using document root <none> and password file <none>
I20260806 15:03:19.970578 6482 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:19.970615 6482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:19.970701 6482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:19.971585 6482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-1/data/instance:
uuid: "35c6ddfac6a94a6d849f69178ee51868"
format_stamp: "Formatted at 2026-08-06 15:03:19 on dist-test-slave-tc2s"
server_key: "e13f45f47d63118651f795aa5e700f33"
server_key_iv: "db841a50b2136dc16bb492f309ce02bf"
server_key_version: "encryptionkey@0"
I20260806 15:03:19.971851 6482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-1/wal/instance:
uuid: "35c6ddfac6a94a6d849f69178ee51868"
format_stamp: "Formatted at 2026-08-06 15:03:19 on dist-test-slave-tc2s"
server_key: "e13f45f47d63118651f795aa5e700f33"
server_key_iv: "db841a50b2136dc16bb492f309ce02bf"
server_key_version: "encryptionkey@0"
I20260806 15:03:19.972947 6482 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:03:19.973532 6501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:19.973665 6482 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:19.973743 6482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "35c6ddfac6a94a6d849f69178ee51868"
format_stamp: "Formatted at 2026-08-06 15:03:19 on dist-test-slave-tc2s"
server_key: "e13f45f47d63118651f795aa5e700f33"
server_key_iv: "db841a50b2136dc16bb492f309ce02bf"
server_key_version: "encryptionkey@0"
I20260806 15:03:19.973798 6482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:19.983882 6482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:19.985762 6482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:19.985883 6482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:19.986094 6482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:19.986356 6482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:19.986389 6482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:19.986423 6482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:19.986449 6482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:19.991770 6482 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:43191
I20260806 15:03:19.991828 6667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:43191 every 8 connection(s)
I20260806 15:03:19.992076 6482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
I20260806 15:03:19.994689 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 6482
I20260806 15:03:19.994805 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-1/wal/instance
I20260806 15:03:19.994974 945 external_mini_cluster.cc:1428] Setting key cb156fde57493bac7bddbf80745a2519
Aug 06 15:03:19 dist-test-slave-tc2s krb5kdc[6110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028599, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:20.001096 6233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:19.992691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:41357 (local address 127.0.236.126:44569)
0806 15:03:19.992752 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:19.992754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:19.992899 (+ 145us) server_negotiation.cc:407] Connection header received
0806 15:03:19.993702 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:19.993703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:19.993712 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:19.993761 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:19.994286 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:19.994931 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:19.996114 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:19.996382 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:19.996452 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:19.998285 (+ 1833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:19.998286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:19.998286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:19.998294 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:19.999687 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:20.000115 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:20.000116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:20.000116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:20.000181 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:20.000390 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:20.000391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:20.000391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:20.000525 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:20.000592 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:20.000962 (+ 370us) server_negotiation.cc:299] Negotiation successful
0806 15:03:20.001033 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 15:03:20.001312 6672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:19.992649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:44569 (local address 127.0.236.66:41357)
0806 15:03:19.992791 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:19.992828 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:19.993625 (+ 797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:19.993808 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:19.993811 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:19.994056 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:19.994223 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:19.994227 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:19.995023 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:19.995024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:19.996034 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:19.996036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:19.996456 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:19.996457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:19.996546 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:19.996962 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:19.996970 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:19.998202 (+ 1232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:19.999798 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:19.999805 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:19.999807 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:20.000046 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:20.000239 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:20.000240 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:20.000240 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:20.000336 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:20.000627 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:20.000634 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:20.000795 (+ 161us) client_negotiation.cc:769] Sending connection context
0806 15:03:20.000927 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 15:03:20.001098 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":77,"threads_started":1}
I20260806 15:03:20.001760 6670 heartbeater.cc:347] Connected to a master server at 127.0.236.126:44569
I20260806 15:03:20.001842 6670 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:20.002027 6670 heartbeater.cc:511] Master 127.0.236.126:44569 requested a full tablet report, sending...
I20260806 15:03:20.002383 6178 ts_manager.cc:194] Registered new tserver with Master: 35c6ddfac6a94a6d849f69178ee51868 (127.0.236.66:43191)
I20260806 15:03:20.002868 6178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.66@KRBTEST.COM'} at 127.0.236.66:41357
WARNING: no policy specified for kudu/127.0.236.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
I20260806 15:03:20.018250 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:0
--local_ip_for_outbound_sockets=127.0.236.67
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44569
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:34141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:20.095355 6678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:20.095539 6678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:20.095561 6678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:20.096621 6678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:20.096652 6678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:20.096675 6678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:03:20.098066 6678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:34141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:20.098232 6678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:20.098392 6678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:20.102968 6693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:20.103086 6678 server_base.cc:1034] running on GCE node
W20260806 15:03:20.102978 6687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:20.102988 6688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:20.103370 6678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:20.103591 6678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:20.104727 6678 hybrid_clock.cc:648] HybridClock initialized: now 1786028600104705 us; error 35 us; skew 500 ppm
Aug 06 15:03:20 dist-test-slave-tc2s krb5kdc[6110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028600, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:20.107002 6678 init.cc:377] Logged in from keytab as kudu/127.0.236.67@KRBTEST.COM (short username kudu)
I20260806 15:03:20.107407 6678 webserver.cc:460] Webserver started at http://127.0.236.67:46835/ using document root <none> and password file <none>
I20260806 15:03:20.107594 6678 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:20.107641 6678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:20.107764 6678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:20.108707 6678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-2/data/instance:
uuid: "e3fb768b067a45deaee365a6876ea127"
format_stamp: "Formatted at 2026-08-06 15:03:20 on dist-test-slave-tc2s"
server_key: "a865177f8ffceb822644210c022c9d58"
server_key_iv: "6ca505dd1fb7aa9ebc4c9adb4ac72a71"
server_key_version: "encryptionkey@0"
I20260806 15:03:20.109040 6678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-2/wal/instance:
uuid: "e3fb768b067a45deaee365a6876ea127"
format_stamp: "Formatted at 2026-08-06 15:03:20 on dist-test-slave-tc2s"
server_key: "a865177f8ffceb822644210c022c9d58"
server_key_iv: "6ca505dd1fb7aa9ebc4c9adb4ac72a71"
server_key_version: "encryptionkey@0"
I20260806 15:03:20.110237 6678 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:03:20.110852 6703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:20.110975 6678 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:20.111055 6678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "e3fb768b067a45deaee365a6876ea127"
format_stamp: "Formatted at 2026-08-06 15:03:20 on dist-test-slave-tc2s"
server_key: "a865177f8ffceb822644210c022c9d58"
server_key_iv: "6ca505dd1fb7aa9ebc4c9adb4ac72a71"
server_key_version: "encryptionkey@0"
I20260806 15:03:20.111112 6678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:20.122869 6678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:20.124395 6678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:20.124516 6678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:20.124703 6678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:20.124963 6678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:20.124996 6678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:20.125022 6678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:20.125047 6678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:20.130292 6678 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:42639
I20260806 15:03:20.130327 6881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:42639 every 8 connection(s)
I20260806 15:03:20.130581 6678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
I20260806 15:03:20.131961 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 6678
I20260806 15:03:20.132045 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028580791223-945-0/minicluster-data/ts-2/wal/instance
I20260806 15:03:20.132200 945 external_mini_cluster.cc:1428] Setting key 824f3d55a5d6c1a80c6e0b262806b772
Aug 06 15:03:20 dist-test-slave-tc2s krb5kdc[6110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028600, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:20.138718 6233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:20.131226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:46927 (local address 127.0.236.126:44569)
0806 15:03:20.131298 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:20.131300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:20.131419 (+ 119us) server_negotiation.cc:407] Connection header received
0806 15:03:20.132263 (+ 844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:20.132265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:20.132277 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:20.132340 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:20.133137 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:20.133619 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:20.134551 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:20.134728 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:20.134773 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:20.136493 (+ 1720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:20.136494 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:20.136494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:20.136501 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:20.137720 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:20.138074 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:20.138075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:20.138075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:20.138124 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:20.138290 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:20.138290 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:20.138290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:20.138375 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:20.138416 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:20.138601 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 15:03:20.138658 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 15:03:20.138890 6886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:20.131185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:44569 (local address 127.0.236.67:46927)
0806 15:03:20.131315 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:20.131346 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:20.132165 (+ 819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:20.132490 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:20.132494 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:20.132810 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:20.133069 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:20.133076 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:20.133679 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:20.133680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:20.134486 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:20.134488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:20.134782 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:20.134782 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:20.134869 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:20.135225 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:20.135232 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:20.136416 (+ 1184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:20.137776 (+ 1360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:20.137778 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:20.137780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:20.138026 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:20.138169 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:20.138170 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:20.138170 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:20.138255 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:20.138413 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:20.138416 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:20.138517 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 15:03:20.138588 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 15:03:20.138690 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":73,"threads_started":1}
I20260806 15:03:20.139309 6882 heartbeater.cc:347] Connected to a master server at 127.0.236.126:44569
I20260806 15:03:20.139390 6882 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:20.139551 6882 heartbeater.cc:511] Master 127.0.236.126:44569 requested a full tablet report, sending...
I20260806 15:03:20.139850 6178 ts_manager.cc:194] Registered new tserver with Master: e3fb768b067a45deaee365a6876ea127 (127.0.236.67:42639)
I20260806 15:03:20.140239 6178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.67@KRBTEST.COM'} at 127.0.236.67:46927
I20260806 15:03:20.143688 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 15:03:20 dist-test-slave-tc2s krb5kdc[6110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 15:03:20 dist-test-slave-tc2s krb5kdc[6110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.65@KRBTEST.COM
I20260806 15:03:20.165032 6893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:20.158408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59070 (local address 127.0.236.65:34409)
0806 15:03:20.158583 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:20.158588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:20.158598 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:03:20.158620 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:20.158621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:20.158634 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:20.158696 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:20.159213 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:20.159885 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:20.160561 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:20.160846 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:20.160890 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:20.162805 (+ 1915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:20.162813 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:20.162813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:20.162820 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:20.164183 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:20.164445 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:20.164447 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:20.164447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:20.164506 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:20.164624 (+ 118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:20.164624 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:20.164624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:20.164730 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:20.164833 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:20.164864 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 15:03:20.164931 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":93,"threads_started":1}
Aug 06 15:03:20 dist-test-slave-tc2s krb5kdc[6110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:20.171516 6233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:20.166150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59260 (local address 127.0.236.126:44569)
0806 15:03:20.166188 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:20.166190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:20.166198 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:20.166235 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:20.166235 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:20.166243 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:20.166281 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:20.166649 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:20.167089 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:20.167659 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:20.167820 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:20.167858 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:20.169565 (+ 1707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:20.169566 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:20.169566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:20.169572 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:20.170740 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:20.170993 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:20.170993 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:20.170994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:20.171035 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:20.171155 (+ 120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:20.171155 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:20.171155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:20.171270 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:20.171334 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:20.171404 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 15:03:20.171454 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 15:03:20.171736 6178 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59260
I20260806 15:03:20.172041 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 6271
I20260806 15:03:20.196590 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 6482
I20260806 15:03:20.203280 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 6678
I20260806 15:03:20.209745 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 6126
2026-08-06T15:03:20Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1839 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:03:20 dist-test-slave-tc2s krb5kdc[6904](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:03:20 dist-test-slave-tc2s krb5kdc[6904](info): set up 2 sockets
Aug 06 15:03:20 dist-test-slave-tc2s krb5kdc[6904](info): commencing operation
krb5kdc: starting...
W20260806 15:03:21.315963 945 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:03:21 dist-test-slave-tc2s krb5kdc[6904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:03:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:03:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:03:21.423985 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:40215
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:41851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:40215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:21.502147 6927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:21.502334 6927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:21.502355 6927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:21.503471 6927 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:03:21.503510 6927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:21.503523 6927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:21.503535 6927 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:03:21.503545 6927 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:03:21.505014 6927 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:41851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:40215
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:40215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:21.505234 6927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:21.505434 6927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:21.509856 6937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:21.509990 6939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:21.509943 6936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:21.510082 6927 server_base.cc:1034] running on GCE node
I20260806 15:03:21.510396 6927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:21.510680 6927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:21.511812 6927 hybrid_clock.cc:648] HybridClock initialized: now 1786028601511800 us; error 24 us; skew 500 ppm
Aug 06 15:03:21 dist-test-slave-tc2s krb5kdc[6904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028601, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:21.514137 6927 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:03:21.514454 6927 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:21.514505 6927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:21.514627 6927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:21.515599 6927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "c9741a25582540c79d33470f8d7b3cfa"
format_stamp: "Formatted at 2026-08-06 15:03:21 on dist-test-slave-tc2s"
server_key: "9df3a38d434ec1f9f954fe1507fb2cf6"
server_key_iv: "7bb2f07c3b4a87ed84ea9725b2c18f6d"
server_key_version: "encryptionkey@0"
I20260806 15:03:21.515906 6927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "c9741a25582540c79d33470f8d7b3cfa"
format_stamp: "Formatted at 2026-08-06 15:03:21 on dist-test-slave-tc2s"
server_key: "9df3a38d434ec1f9f954fe1507fb2cf6"
server_key_iv: "7bb2f07c3b4a87ed84ea9725b2c18f6d"
server_key_version: "encryptionkey@0"
I20260806 15:03:21.517040 6927 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:03:21.517627 6946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:21.517773 6927 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:21.517854 6927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "c9741a25582540c79d33470f8d7b3cfa"
format_stamp: "Formatted at 2026-08-06 15:03:21 on dist-test-slave-tc2s"
server_key: "9df3a38d434ec1f9f954fe1507fb2cf6"
server_key_iv: "7bb2f07c3b4a87ed84ea9725b2c18f6d"
server_key_version: "encryptionkey@0"
I20260806 15:03:21.517941 6927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:21.529834 6927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:21.531395 6927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:21.531525 6927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:21.535166 6927 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:40215
I20260806 15:03:21.535190 7019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:40215 every 8 connection(s)
I20260806 15:03:21.535430 6927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:03:21.536440 7020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:21.537709 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 6927
I20260806 15:03:21.537825 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:03:21.538046 945 external_mini_cluster.cc:1428] Setting key b7d989a76964ebd3d37ed43f2dd106dc
I20260806 15:03:21.538843 7020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9741a25582540c79d33470f8d7b3cfa: Bootstrap starting.
I20260806 15:03:21.539417 7020 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9741a25582540c79d33470f8d7b3cfa: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:21.539765 7020 log.cc:824] T 00000000000000000000000000000000 P c9741a25582540c79d33470f8d7b3cfa: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:21.540489 7020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9741a25582540c79d33470f8d7b3cfa: No bootstrap required, opened a new log
I20260806 15:03:21.542464 7020 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9741a25582540c79d33470f8d7b3cfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9741a25582540c79d33470f8d7b3cfa" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 40215 } }
I20260806 15:03:21.542615 7020 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9741a25582540c79d33470f8d7b3cfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:21.542650 7020 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9741a25582540c79d33470f8d7b3cfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9741a25582540c79d33470f8d7b3cfa, State: Initialized, Role: FOLLOWER
I20260806 15:03:21.542778 7020 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9741a25582540c79d33470f8d7b3cfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9741a25582540c79d33470f8d7b3cfa" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 40215 } }
I20260806 15:03:21.542857 7020 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9741a25582540c79d33470f8d7b3cfa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:03:21.542881 7020 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9741a25582540c79d33470f8d7b3cfa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:03:21.542927 7020 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9741a25582540c79d33470f8d7b3cfa [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:21.543567 7020 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9741a25582540c79d33470f8d7b3cfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9741a25582540c79d33470f8d7b3cfa" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 40215 } }
I20260806 15:03:21.543682 7020 leader_election.cc:302] T 00000000000000000000000000000000 P c9741a25582540c79d33470f8d7b3cfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9741a25582540c79d33470f8d7b3cfa; no voters:
I20260806 15:03:21.543843 7020 leader_election.cc:288] T 00000000000000000000000000000000 P c9741a25582540c79d33470f8d7b3cfa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:03:21.543888 7027 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9741a25582540c79d33470f8d7b3cfa [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:21.544085 7020 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9741a25582540c79d33470f8d7b3cfa [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 15:03:21 dist-test-slave-tc2s krb5kdc[6904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:21.544096 7027 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9741a25582540c79d33470f8d7b3cfa [term 1 LEADER]: Becoming Leader. State: Replica: c9741a25582540c79d33470f8d7b3cfa, State: Running, Role: LEADER
I20260806 15:03:21.545426 7027 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9741a25582540c79d33470f8d7b3cfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9741a25582540c79d33470f8d7b3cfa" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 40215 } }
I20260806 15:03:21.545990 7029 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9741a25582540c79d33470f8d7b3cfa [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9741a25582540c79d33470f8d7b3cfa. Latest consensus state: current_term: 1 leader_uuid: "c9741a25582540c79d33470f8d7b3cfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9741a25582540c79d33470f8d7b3cfa" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 40215 } } }
I20260806 15:03:21.546010 7028 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9741a25582540c79d33470f8d7b3cfa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9741a25582540c79d33470f8d7b3cfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9741a25582540c79d33470f8d7b3cfa" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 40215 } } }
I20260806 15:03:21.546728 7029 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9741a25582540c79d33470f8d7b3cfa [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:21.546736 7028 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9741a25582540c79d33470f8d7b3cfa [sys.catalog]: This master's current role is: LEADER
W20260806 15:03:21.546465 7050 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c9741a25582540c79d33470f8d7b3cfa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:03:21.546967 7050 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:03:21.547055 7054 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:03:21.547219 7054 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:03:21.548981 7024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:21.538559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60938 (local address 127.0.236.126:40215)
0806 15:03:21.538715 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:21.538724 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:21.538749 (+ 25us) server_negotiation.cc:407] Connection header received
0806 15:03:21.539163 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:21.539170 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:21.539215 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:21.539325 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:21.540062 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:21.541071 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:21.542128 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:21.542518 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:21.542594 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:21.545272 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:21.545276 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:21.545276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:21.545286 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:21.547467 (+ 2181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:21.547861 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:21.547863 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:21.547864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:21.547971 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:21.548138 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:21.548138 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:21.548139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:21.548432 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:21.548578 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:21.548650 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 15:03:21.548782 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
I20260806 15:03:21.550786 7054 catalog_manager.cc:1347] Generated new cluster ID: 95ec772c783a48ffa3c92fbca5067966
I20260806 15:03:21.550830 7054 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:03:21.562909 7054 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:03:21.563380 7054 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:03:21.571655 7054 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9741a25582540c79d33470f8d7b3cfa: Generated new TSK 0
I20260806 15:03:21.571789 7054 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.236.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
I20260806 15:03:21.600457 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:0
--local_ip_for_outbound_sockets=127.0.236.65
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:40215
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:41851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:21.684602 7062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:21.684784 7062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:21.684806 7062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:21.685920 7062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:21.685956 7062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:21.685985 7062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:03:21.687434 7062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:41851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:40215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:21.687633 7062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:21.687824 7062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:21.692337 7075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:21.692332 7071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:21.692335 7072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:21.692509 7062 server_base.cc:1034] running on GCE node
I20260806 15:03:21.692754 7062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:21.692988 7062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:21.694118 7062 hybrid_clock.cc:648] HybridClock initialized: now 1786028601694104 us; error 31 us; skew 500 ppm
Aug 06 15:03:21 dist-test-slave-tc2s krb5kdc[6904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028601, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:21.696758 7062 init.cc:377] Logged in from keytab as kudu/127.0.236.65@KRBTEST.COM (short username kudu)
I20260806 15:03:21.697052 7062 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:21.697098 7062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:21.697209 7062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:21.698169 7062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-0/data/instance:
uuid: "84bcfcb7bf89438591ec47d68d7fdd2d"
format_stamp: "Formatted at 2026-08-06 15:03:21 on dist-test-slave-tc2s"
server_key: "b7fea7565a37ee2f4e9298e38b0d055e"
server_key_iv: "53a2854c59253d44574b06c143ff700d"
server_key_version: "encryptionkey@0"
I20260806 15:03:21.698465 7062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-0/wal/instance:
uuid: "84bcfcb7bf89438591ec47d68d7fdd2d"
format_stamp: "Formatted at 2026-08-06 15:03:21 on dist-test-slave-tc2s"
server_key: "b7fea7565a37ee2f4e9298e38b0d055e"
server_key_iv: "53a2854c59253d44574b06c143ff700d"
server_key_version: "encryptionkey@0"
I20260806 15:03:21.699594 7062 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:21.700194 7085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:21.700342 7062 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:21.700415 7062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "84bcfcb7bf89438591ec47d68d7fdd2d"
format_stamp: "Formatted at 2026-08-06 15:03:21 on dist-test-slave-tc2s"
server_key: "b7fea7565a37ee2f4e9298e38b0d055e"
server_key_iv: "53a2854c59253d44574b06c143ff700d"
server_key_version: "encryptionkey@0"
I20260806 15:03:21.700476 7062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:21.723448 7062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:21.725010 7062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:21.725127 7062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:21.725324 7062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:21.725602 7062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:21.725634 7062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:21.725660 7062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:21.725684 7062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:21.730957 7062 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:45979
I20260806 15:03:21.731010 7260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:45979 every 8 connection(s)
I20260806 15:03:21.731226 7062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:03:21.735193 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 7062
I20260806 15:03:21.735307 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-0/wal/instance
I20260806 15:03:21.735502 945 external_mini_cluster.cc:1428] Setting key 9dd48d7c701dc40564b8b2c9a1272f74
Aug 06 15:03:21 dist-test-slave-tc2s krb5kdc[6904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028601, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:21.739748 7024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:21.731970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:50437 (local address 127.0.236.126:40215)
0806 15:03:21.732042 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:21.732044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:21.732184 (+ 140us) server_negotiation.cc:407] Connection header received
0806 15:03:21.732605 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:21.732606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:21.732614 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:21.732664 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:21.733485 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:21.733981 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:21.734876 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:21.735060 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:21.735100 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:21.737281 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:21.737282 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:21.737283 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:21.737290 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:21.738536 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:21.738891 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:21.738893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:21.738893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:21.738946 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:21.739161 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:21.739162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:21.739162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:21.739267 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:21.739322 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:21.739578 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 15:03:21.739660 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 15:03:21.739898 7264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:21.731898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:40215 (local address 127.0.236.65:50437)
0806 15:03:21.732069 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:21.732109 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:21.732536 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:21.732715 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:21.732718 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:21.732954 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:21.733421 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:21.733426 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:21.734034 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:21.734035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:21.734817 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:21.734819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:21.735124 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:21.735125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:21.735261 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:21.735771 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:21.735780 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:21.737191 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:21.738611 (+ 1420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:21.738619 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:21.738621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:21.738825 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:21.739001 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:21.739002 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:21.739002 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:21.739105 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:21.739323 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:21.739330 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:21.739461 (+ 131us) client_negotiation.cc:769] Sending connection context
0806 15:03:21.739547 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 15:03:21.739691 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":71,"threads_started":1}
I20260806 15:03:21.740406 7261 heartbeater.cc:347] Connected to a master server at 127.0.236.126:40215
I20260806 15:03:21.740492 7261 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:21.740633 7261 heartbeater.cc:511] Master 127.0.236.126:40215 requested a full tablet report, sending...
I20260806 15:03:21.741214 6965 ts_manager.cc:194] Registered new tserver with Master: 84bcfcb7bf89438591ec47d68d7fdd2d (127.0.236.65:45979)
I20260806 15:03:21.742130 6965 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.65@KRBTEST.COM'} at 127.0.236.65:50437
WARNING: no policy specified for kudu/127.0.236.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
I20260806 15:03:21.758167 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:0
--local_ip_for_outbound_sockets=127.0.236.66
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:40215
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:41851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:21.836297 7268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:21.836477 7268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:21.836498 7268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:21.837554 7268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:21.837584 7268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:21.837610 7268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:03:21.839032 7268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:41851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:40215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:21.839195 7268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:21.839359 7268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:21.843802 7282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:21.843924 7268 server_base.cc:1034] running on GCE node
W20260806 15:03:21.843816 7279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:21.843999 7276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:21.844209 7268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:21.844450 7268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:21.845580 7268 hybrid_clock.cc:648] HybridClock initialized: now 1786028601845566 us; error 27 us; skew 500 ppm
Aug 06 15:03:21 dist-test-slave-tc2s krb5kdc[6904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028601, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:21.847965 7268 init.cc:377] Logged in from keytab as kudu/127.0.236.66@KRBTEST.COM (short username kudu)
I20260806 15:03:21.848259 7268 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:21.848308 7268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:21.848440 7268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:21.849371 7268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-1/data/instance:
uuid: "709ada1eccfe441397baf0e242e447ce"
format_stamp: "Formatted at 2026-08-06 15:03:21 on dist-test-slave-tc2s"
server_key: "36c4848d6f4c1367260ecdff59fd8e50"
server_key_iv: "2a49157754a582e3186b6c7f0bc1f540"
server_key_version: "encryptionkey@0"
I20260806 15:03:21.849702 7268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-1/wal/instance:
uuid: "709ada1eccfe441397baf0e242e447ce"
format_stamp: "Formatted at 2026-08-06 15:03:21 on dist-test-slave-tc2s"
server_key: "36c4848d6f4c1367260ecdff59fd8e50"
server_key_iv: "2a49157754a582e3186b6c7f0bc1f540"
server_key_version: "encryptionkey@0"
I20260806 15:03:21.850878 7268 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 15:03:21.851501 7291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:21.851649 7268 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:21.851727 7268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "709ada1eccfe441397baf0e242e447ce"
format_stamp: "Formatted at 2026-08-06 15:03:21 on dist-test-slave-tc2s"
server_key: "36c4848d6f4c1367260ecdff59fd8e50"
server_key_iv: "2a49157754a582e3186b6c7f0bc1f540"
server_key_version: "encryptionkey@0"
I20260806 15:03:21.851791 7268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:21.881798 7268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:21.883435 7268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:21.883538 7268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:21.883711 7268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:21.883945 7268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:21.883975 7268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:21.883994 7268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:21.884007 7268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:21.889144 7268 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:42709
I20260806 15:03:21.889218 7467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:42709 every 8 connection(s)
I20260806 15:03:21.889436 7268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
I20260806 15:03:21.892398 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 7268
I20260806 15:03:21.892488 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-1/wal/instance
I20260806 15:03:21.892665 945 external_mini_cluster.cc:1428] Setting key 1ceeaea74566394d0c24e7d573d7a47a
Aug 06 15:03:21 dist-test-slave-tc2s krb5kdc[6904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028601, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:21.898557 7024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:21.890136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:60263 (local address 127.0.236.126:40215)
0806 15:03:21.890198 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:21.890200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:21.890376 (+ 176us) server_negotiation.cc:407] Connection header received
0806 15:03:21.890827 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:21.890829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:21.890842 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:21.890892 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:21.891762 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:21.892263 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:21.893504 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:21.893710 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:21.893765 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:21.895892 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:21.895893 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:21.895893 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:21.895900 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:21.897179 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:21.897542 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:21.897544 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:21.897545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:21.897612 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:21.897836 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:21.897837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:21.897837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:21.897996 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:21.898065 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:21.898419 (+ 354us) server_negotiation.cc:299] Negotiation successful
0806 15:03:21.898494 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 15:03:21.898777 7473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:21.890110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:40215 (local address 127.0.236.66:60263)
0806 15:03:21.890265 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:21.890297 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:21.890730 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:21.890961 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:21.890965 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:21.891199 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:21.891663 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:21.891668 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:21.892346 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:21.892347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:21.893425 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:21.893428 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:21.893776 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:21.893777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:21.893924 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:21.894375 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:21.894385 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:21.895792 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:21.897254 (+ 1462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:21.897260 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:21.897261 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:21.897463 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:21.897683 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:21.897684 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:21.897684 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:21.897782 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:21.898097 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:21.898104 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:21.898264 (+ 160us) client_negotiation.cc:769] Sending connection context
0806 15:03:21.898395 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 15:03:21.898560 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":83,"threads_started":1}
I20260806 15:03:21.899210 7469 heartbeater.cc:347] Connected to a master server at 127.0.236.126:40215
I20260806 15:03:21.899293 7469 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:21.899442 7469 heartbeater.cc:511] Master 127.0.236.126:40215 requested a full tablet report, sending...
I20260806 15:03:21.899827 6965 ts_manager.cc:194] Registered new tserver with Master: 709ada1eccfe441397baf0e242e447ce (127.0.236.66:42709)
I20260806 15:03:21.900182 6965 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.66@KRBTEST.COM'} at 127.0.236.66:60263
WARNING: no policy specified for kudu/127.0.236.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
I20260806 15:03:21.916682 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:0
--local_ip_for_outbound_sockets=127.0.236.67
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:40215
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:41851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:21.994905 7477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:21.995095 7477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:21.995118 7477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:21.996212 7477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:21.996251 7477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:21.996279 7477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:03:21.997727 7477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:41851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:40215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:21.997962 7477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:21.998158 7477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:22.002710 7486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:22.002691 7488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:22.002677 7492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:22.002853 7477 server_base.cc:1034] running on GCE node
I20260806 15:03:22.003221 7477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:22.003448 7477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:22.004580 7477 hybrid_clock.cc:648] HybridClock initialized: now 1786028602004567 us; error 29 us; skew 500 ppm
Aug 06 15:03:22 dist-test-slave-tc2s krb5kdc[6904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028602, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:22.006945 7477 init.cc:377] Logged in from keytab as kudu/127.0.236.67@KRBTEST.COM (short username kudu)
I20260806 15:03:22.007236 7477 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:22.007278 7477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:22.007392 7477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:22.008338 7477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-2/data/instance:
uuid: "43bd0dd954344685be35f76ee0857d2c"
format_stamp: "Formatted at 2026-08-06 15:03:22 on dist-test-slave-tc2s"
server_key: "5f554a31901fdf76e0b11dfe0fadc1b0"
server_key_iv: "139d69f44f291ccfe8e8caac43219377"
server_key_version: "encryptionkey@0"
I20260806 15:03:22.008638 7477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-2/wal/instance:
uuid: "43bd0dd954344685be35f76ee0857d2c"
format_stamp: "Formatted at 2026-08-06 15:03:22 on dist-test-slave-tc2s"
server_key: "5f554a31901fdf76e0b11dfe0fadc1b0"
server_key_iv: "139d69f44f291ccfe8e8caac43219377"
server_key_version: "encryptionkey@0"
I20260806 15:03:22.009805 7477 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:03:22.010516 7501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:22.010667 7477 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:03:22.010749 7477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "43bd0dd954344685be35f76ee0857d2c"
format_stamp: "Formatted at 2026-08-06 15:03:22 on dist-test-slave-tc2s"
server_key: "5f554a31901fdf76e0b11dfe0fadc1b0"
server_key_iv: "139d69f44f291ccfe8e8caac43219377"
server_key_version: "encryptionkey@0"
I20260806 15:03:22.010816 7477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:22.022194 7477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:22.024143 7477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:22.024269 7477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:22.024462 7477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:22.024734 7477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:22.024765 7477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:22.024792 7477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:22.024816 7477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:22.030085 7477 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:38443
I20260806 15:03:22.030139 7671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:38443 every 8 connection(s)
I20260806 15:03:22.030354 7477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
I20260806 15:03:22.030864 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 7477
I20260806 15:03:22.030977 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-2/wal/instance
I20260806 15:03:22.031183 945 external_mini_cluster.cc:1428] Setting key 757f601bba35f55cca9b37d42587eb9a
Aug 06 15:03:22 dist-test-slave-tc2s krb5kdc[6904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028602, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:22.039024 7024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.031069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:60657 (local address 127.0.236.126:40215)
0806 15:03:22.031112 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:22.031115 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:22.031290 (+ 175us) server_negotiation.cc:407] Connection header received
0806 15:03:22.031826 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:22.031827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:22.031838 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:22.031873 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:22.032765 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.033252 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.034272 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.034554 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.034721 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:22.036599 (+ 1878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:22.036600 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:22.036600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:22.036608 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:22.037719 (+ 1111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:22.038131 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:22.038132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:22.038132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:22.038190 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:22.038416 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:22.038417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:22.038417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:22.038530 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:22.038584 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:22.038831 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 15:03:22.038936 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:03:22.039142 7674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.031033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:40215 (local address 127.0.236.67:60657)
0806 15:03:22.031185 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:22.031217 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:22.031737 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:22.031935 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:22.031939 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:22.032219 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:22.032674 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:22.032679 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.033330 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.033331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:22.034196 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:22.034199 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.034730 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.034731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:22.034861 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:22.035238 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:22.035246 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:22.036512 (+ 1266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:22.037794 (+ 1282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:22.037801 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:22.037803 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:22.038064 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:22.038243 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:22.038244 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:22.038244 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:22.038353 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:22.038584 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:22.038591 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:22.038716 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 15:03:22.038800 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 15:03:22.038943 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260806 15:03:22.039574 7672 heartbeater.cc:347] Connected to a master server at 127.0.236.126:40215
I20260806 15:03:22.039650 7672 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:22.039784 7672 heartbeater.cc:511] Master 127.0.236.126:40215 requested a full tablet report, sending...
I20260806 15:03:22.040125 6965 ts_manager.cc:194] Registered new tserver with Master: 43bd0dd954344685be35f76ee0857d2c (127.0.236.67:38443)
I20260806 15:03:22.040493 6965 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.67@KRBTEST.COM'} at 127.0.236.67:60657
I20260806 15:03:22.042721 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:03:22.049211 7024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.043924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60950 (local address 127.0.236.126:40215)
0806 15:03:22.043972 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:22.043974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:22.044064 (+ 90us) server_negotiation.cc:407] Connection header received
0806 15:03:22.044131 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:22.044132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:22.044141 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:22.044190 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:22.044740 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.045242 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.045964 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.046205 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.046260 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:22.047250 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:22.047251 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:22.047252 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:22.047258 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:22.048231 (+ 973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:22.048532 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:22.048533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:22.048533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:22.048586 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:22.048736 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:22.048737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:22.048737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:22.048868 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:22.048965 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:22.049058 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 15:03:22.049128 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 15:03:22.050458 6965 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:03:22.052078 6965 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:03:22.057696 7686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.054030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:45979 (local address 127.0.0.1:33884)
0806 15:03:22.054399 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:22.054407 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:22.054473 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:22.054657 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:22.054661 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:22.054663 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:22.054724 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:22.054725 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.055820 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.055821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:22.056630 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:22.056632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.057418 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.057419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:22.057503 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:22.057515 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 15:03:22.057574 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 15:03:22.057624 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":4,"thread_start_us":60,"threads_started":1}
I20260806 15:03:22.057713 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.054208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33884 (local address 127.0.236.65:45979)
0806 15:03:22.054360 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:22.054364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:22.054461 (+ 97us) server_negotiation.cc:407] Connection header received
0806 15:03:22.054546 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:22.054548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:22.054564 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:22.054649 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:22.054849 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.055736 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.056711 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.057331 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.057395 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:22.057454 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:22.057595 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 15:03:22.057630 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":70,"threads_started":1}
I20260806 15:03:22.058327 7154 tablet_service.cc:1467] Processing CreateTablet for tablet 4d04e04c39e648e0a8ef4ea4ef74dcb4 (DEFAULT_TABLE table=test-table [id=7ae725599b9a4627a53e011f560e68a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:22.058524 7689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.054222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:42709 (local address 127.0.0.1:34748)
0806 15:03:22.054592 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:22.054601 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:22.054648 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:22.055010 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:22.055011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:22.055013 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:22.055045 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:22.055046 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.056250 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.056250 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:22.057364 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:22.057366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.058290 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.058292 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:22.058358 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:22.058362 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:03:22.058414 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 15:03:22.058454 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":47,"threads_started":1}
I20260806 15:03:22.058655 7684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.054029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:38443 (local address 127.0.0.1:58472)
0806 15:03:22.054210 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:22.054249 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:22.054307 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:22.054771 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:22.054772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:22.054774 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:22.054821 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:22.054822 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.056064 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.056065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:22.057226 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:22.057229 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.058458 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.058458 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:22.058515 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:22.058518 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:03:22.058563 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 15:03:22.058598 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":24,"thread_start_us":63,"threads_started":1}
I20260806 15:03:22.058665 7154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d04e04c39e648e0a8ef4ea4ef74dcb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:22.058795 7692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.054410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34748 (local address 127.0.236.66:42709)
0806 15:03:22.054586 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:22.054592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:22.054699 (+ 107us) server_negotiation.cc:407] Connection header received
0806 15:03:22.054739 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:22.054741 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:22.054788 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:22.054850 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:22.055124 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.056154 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.057439 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.058138 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.058204 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:22.058255 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:22.058682 (+ 427us) server_negotiation.cc:299] Negotiation successful
0806 15:03:22.058721 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":94,"threads_started":1}
I20260806 15:03:22.058995 7688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.054090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58472 (local address 127.0.236.67:38443)
0806 15:03:22.054281 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:22.054285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:22.054296 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:03:22.054593 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:22.054596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:22.054616 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:22.054667 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:22.054895 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.055955 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.057317 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.058303 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.058342 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:22.058387 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:22.058823 (+ 436us) server_negotiation.cc:299] Negotiation successful
0806 15:03:22.058927 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":90,"threads_started":1}
I20260806 15:03:22.059345 7371 tablet_service.cc:1467] Processing CreateTablet for tablet 4d04e04c39e648e0a8ef4ea4ef74dcb4 (DEFAULT_TABLE table=test-table [id=7ae725599b9a4627a53e011f560e68a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:22.059590 7371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d04e04c39e648e0a8ef4ea4ef74dcb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:22.059602 7578 tablet_service.cc:1467] Processing CreateTablet for tablet 4d04e04c39e648e0a8ef4ea4ef74dcb4 (DEFAULT_TABLE table=test-table [id=7ae725599b9a4627a53e011f560e68a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:22.059912 7578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d04e04c39e648e0a8ef4ea4ef74dcb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:22.061777 7693 tablet_bootstrap.cc:492] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 709ada1eccfe441397baf0e242e447ce: Bootstrap starting.
I20260806 15:03:22.062244 7695 tablet_bootstrap.cc:492] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 84bcfcb7bf89438591ec47d68d7fdd2d: Bootstrap starting.
I20260806 15:03:22.062431 7693 tablet_bootstrap.cc:654] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 709ada1eccfe441397baf0e242e447ce: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:22.062870 7695 tablet_bootstrap.cc:654] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 84bcfcb7bf89438591ec47d68d7fdd2d: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:22.063048 7700 tablet_bootstrap.cc:492] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c: Bootstrap starting.
I20260806 15:03:22.063182 7693 log.cc:824] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 709ada1eccfe441397baf0e242e447ce: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:22.063445 7695 log.cc:824] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 84bcfcb7bf89438591ec47d68d7fdd2d: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:22.063628 7700 tablet_bootstrap.cc:654] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:22.063947 7693 tablet_bootstrap.cc:492] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 709ada1eccfe441397baf0e242e447ce: No bootstrap required, opened a new log
I20260806 15:03:22.064008 7693 ts_tablet_manager.cc:1397] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 709ada1eccfe441397baf0e242e447ce: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:03:22.064107 7700 log.cc:824] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:22.064785 7700 tablet_bootstrap.cc:492] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c: No bootstrap required, opened a new log
I20260806 15:03:22.064841 7700 ts_tablet_manager.cc:1397] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:03:22.065608 7693 raft_consensus.cc:348] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 709ada1eccfe441397baf0e242e447ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcfcb7bf89438591ec47d68d7fdd2d" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45979 } } peers { permanent_uuid: "43bd0dd954344685be35f76ee0857d2c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38443 } } peers { permanent_uuid: "709ada1eccfe441397baf0e242e447ce" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42709 } }
I20260806 15:03:22.065735 7693 raft_consensus.cc:374] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 709ada1eccfe441397baf0e242e447ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:22.065766 7693 raft_consensus.cc:729] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 709ada1eccfe441397baf0e242e447ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 709ada1eccfe441397baf0e242e447ce, State: Initialized, Role: FOLLOWER
I20260806 15:03:22.065879 7693 consensus_queue.cc:260] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 709ada1eccfe441397baf0e242e447ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcfcb7bf89438591ec47d68d7fdd2d" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45979 } } peers { permanent_uuid: "43bd0dd954344685be35f76ee0857d2c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38443 } } peers { permanent_uuid: "709ada1eccfe441397baf0e242e447ce" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42709 } }
I20260806 15:03:22.066197 7693 ts_tablet_manager.cc:1428] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 709ada1eccfe441397baf0e242e447ce: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 15:03:22.066325 7469 heartbeater.cc:503] Master 127.0.236.126:40215 was elected leader, sending a full tablet report...
I20260806 15:03:22.066390 7695 tablet_bootstrap.cc:492] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 84bcfcb7bf89438591ec47d68d7fdd2d: No bootstrap required, opened a new log
I20260806 15:03:22.066303 7700 raft_consensus.cc:348] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcfcb7bf89438591ec47d68d7fdd2d" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45979 } } peers { permanent_uuid: "43bd0dd954344685be35f76ee0857d2c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38443 } } peers { permanent_uuid: "709ada1eccfe441397baf0e242e447ce" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42709 } }
I20260806 15:03:22.066438 7700 raft_consensus.cc:374] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:22.066452 7695 ts_tablet_manager.cc:1397] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 84bcfcb7bf89438591ec47d68d7fdd2d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260806 15:03:22.066481 7700 raft_consensus.cc:729] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43bd0dd954344685be35f76ee0857d2c, State: Initialized, Role: FOLLOWER
I20260806 15:03:22.066592 7700 consensus_queue.cc:260] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcfcb7bf89438591ec47d68d7fdd2d" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45979 } } peers { permanent_uuid: "43bd0dd954344685be35f76ee0857d2c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38443 } } peers { permanent_uuid: "709ada1eccfe441397baf0e242e447ce" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42709 } }
I20260806 15:03:22.066841 7700 ts_tablet_manager.cc:1428] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:03:22.066848 7672 heartbeater.cc:503] Master 127.0.236.126:40215 was elected leader, sending a full tablet report...
I20260806 15:03:22.068195 7695 raft_consensus.cc:348] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 84bcfcb7bf89438591ec47d68d7fdd2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcfcb7bf89438591ec47d68d7fdd2d" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45979 } } peers { permanent_uuid: "43bd0dd954344685be35f76ee0857d2c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38443 } } peers { permanent_uuid: "709ada1eccfe441397baf0e242e447ce" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42709 } }
I20260806 15:03:22.068320 7695 raft_consensus.cc:374] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 84bcfcb7bf89438591ec47d68d7fdd2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:22.068361 7695 raft_consensus.cc:729] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 84bcfcb7bf89438591ec47d68d7fdd2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84bcfcb7bf89438591ec47d68d7fdd2d, State: Initialized, Role: FOLLOWER
I20260806 15:03:22.068461 7695 consensus_queue.cc:260] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 84bcfcb7bf89438591ec47d68d7fdd2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcfcb7bf89438591ec47d68d7fdd2d" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45979 } } peers { permanent_uuid: "43bd0dd954344685be35f76ee0857d2c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38443 } } peers { permanent_uuid: "709ada1eccfe441397baf0e242e447ce" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42709 } }
I20260806 15:03:22.068794 7695 ts_tablet_manager.cc:1428] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 84bcfcb7bf89438591ec47d68d7fdd2d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 15:03:22.068827 7261 heartbeater.cc:503] Master 127.0.236.126:40215 was elected leader, sending a full tablet report...
I20260806 15:03:22.125072 7707 raft_consensus.cc:482] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:03:22.125183 7707 raft_consensus.cc:504] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcfcb7bf89438591ec47d68d7fdd2d" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45979 } } peers { permanent_uuid: "43bd0dd954344685be35f76ee0857d2c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38443 } } peers { permanent_uuid: "709ada1eccfe441397baf0e242e447ce" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42709 } }
I20260806 15:03:22.125505 7707 leader_election.cc:288] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84bcfcb7bf89438591ec47d68d7fdd2d (127.0.236.65:45979), 709ada1eccfe441397baf0e242e447ce (127.0.236.66:42709)
I20260806 15:03:22.128044 7674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.125650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:45979 (local address 127.0.236.67:44251)
0806 15:03:22.125696 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:22.125707 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:22.125748 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:22.125913 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:22.125914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:22.125916 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:22.125987 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:22.125989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.126579 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.126580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:22.127315 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:22.127317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.127810 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.127811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:22.127883 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:22.127885 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 15:03:22.127930 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 15:03:22.127974 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 15:03:22.128044 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.125714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:44251 (local address 127.0.236.65:45979)
0806 15:03:22.125768 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:22.125770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:22.125780 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:03:22.125806 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:22.125806 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:22.125815 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:22.125872 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:22.126046 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.126511 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.127397 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.127745 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.127803 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:22.127811 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:22.127941 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 15:03:22.127975 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 15:03:22.128304 7692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.125829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:50105 (local address 127.0.236.66:42709)
0806 15:03:22.125915 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:22.125917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:22.126018 (+ 101us) server_negotiation.cc:407] Connection header received
0806 15:03:22.126062 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:22.126063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:22.126071 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:22.126119 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:22.126253 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.126718 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.127614 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.127965 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.128023 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:22.128031 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:22.128204 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 15:03:22.128242 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 15:03:22.128347 7185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d04e04c39e648e0a8ef4ea4ef74dcb4" candidate_uuid: "43bd0dd954344685be35f76ee0857d2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84bcfcb7bf89438591ec47d68d7fdd2d" is_pre_election: true
I20260806 15:03:22.128500 7185 raft_consensus.cc:2454] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 84bcfcb7bf89438591ec47d68d7fdd2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43bd0dd954344685be35f76ee0857d2c in term 0.
I20260806 15:03:22.128551 7711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.125742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:42709 (local address 127.0.236.67:50105)
0806 15:03:22.125958 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:22.125967 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:22.126015 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:22.126126 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:22.126126 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:22.126128 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:22.126196 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:22.126198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.126789 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.126790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:22.127531 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:22.127533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.128034 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.128035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:22.128114 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:22.128117 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:03:22.128426 (+ 309us) client_negotiation.cc:240] Negotiation successful
0806 15:03:22.128472 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":127,"threads_started":1}
I20260806 15:03:22.128804 7504 leader_election.cc:302] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43bd0dd954344685be35f76ee0857d2c, 84bcfcb7bf89438591ec47d68d7fdd2d; no voters:
I20260806 15:03:22.128930 7707 raft_consensus.cc:2781] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:03:22.128994 7707 raft_consensus.cc:482] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:03:22.129017 7707 raft_consensus.cc:3037] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:22.129122 7397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d04e04c39e648e0a8ef4ea4ef74dcb4" candidate_uuid: "43bd0dd954344685be35f76ee0857d2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "709ada1eccfe441397baf0e242e447ce" is_pre_election: true
I20260806 15:03:22.129242 7397 raft_consensus.cc:2454] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 709ada1eccfe441397baf0e242e447ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43bd0dd954344685be35f76ee0857d2c in term 0.
I20260806 15:03:22.129719 7707 raft_consensus.cc:504] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcfcb7bf89438591ec47d68d7fdd2d" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45979 } } peers { permanent_uuid: "43bd0dd954344685be35f76ee0857d2c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38443 } } peers { permanent_uuid: "709ada1eccfe441397baf0e242e447ce" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42709 } }
I20260806 15:03:22.129854 7707 leader_election.cc:288] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c [CANDIDATE]: Term 1 election: Requested vote from peers 84bcfcb7bf89438591ec47d68d7fdd2d (127.0.236.65:45979), 709ada1eccfe441397baf0e242e447ce (127.0.236.66:42709)
I20260806 15:03:22.130023 7185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d04e04c39e648e0a8ef4ea4ef74dcb4" candidate_uuid: "43bd0dd954344685be35f76ee0857d2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84bcfcb7bf89438591ec47d68d7fdd2d"
I20260806 15:03:22.130026 7397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d04e04c39e648e0a8ef4ea4ef74dcb4" candidate_uuid: "43bd0dd954344685be35f76ee0857d2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "709ada1eccfe441397baf0e242e447ce"
I20260806 15:03:22.130092 7397 raft_consensus.cc:3037] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 709ada1eccfe441397baf0e242e447ce [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:22.130091 7185 raft_consensus.cc:3037] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 84bcfcb7bf89438591ec47d68d7fdd2d [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:22.130735 7185 raft_consensus.cc:2454] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 84bcfcb7bf89438591ec47d68d7fdd2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43bd0dd954344685be35f76ee0857d2c in term 1.
I20260806 15:03:22.130795 7397 raft_consensus.cc:2454] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 709ada1eccfe441397baf0e242e447ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43bd0dd954344685be35f76ee0857d2c in term 1.
I20260806 15:03:22.130978 7504 leader_election.cc:302] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43bd0dd954344685be35f76ee0857d2c, 709ada1eccfe441397baf0e242e447ce; no voters:
I20260806 15:03:22.131067 7707 raft_consensus.cc:2781] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:22.131225 7707 raft_consensus.cc:686] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c [term 1 LEADER]: Becoming Leader. State: Replica: 43bd0dd954344685be35f76ee0857d2c, State: Running, Role: LEADER
I20260806 15:03:22.131304 7707 consensus_queue.cc:237] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcfcb7bf89438591ec47d68d7fdd2d" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45979 } } peers { permanent_uuid: "43bd0dd954344685be35f76ee0857d2c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38443 } } peers { permanent_uuid: "709ada1eccfe441397baf0e242e447ce" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42709 } }
I20260806 15:03:22.132081 6964 catalog_manager.cc:5495] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 43bd0dd954344685be35f76ee0857d2c (127.0.236.67). New cstate: current_term: 1 leader_uuid: "43bd0dd954344685be35f76ee0857d2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bcfcb7bf89438591ec47d68d7fdd2d" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43bd0dd954344685be35f76ee0857d2c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "709ada1eccfe441397baf0e242e447ce" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42709 } health_report { overall_health: UNKNOWN } } }
W20260806 15:03:22.139899 7470 tablet.cc:2369] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 709ada1eccfe441397baf0e242e447ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:03:22.151329 7688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.149086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58488 (local address 127.0.236.67:38443)
0806 15:03:22.149141 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:22.149143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:22.149151 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:22.149200 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:22.149201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:22.149219 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:22.149278 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:03:22.149415 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.149856 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.150648 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:22.150832 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:22.150892 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:22.151027 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:03:22.151122 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:03:22.151190 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:22.151227 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 15:03:22.151245 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 15:03:22.152911 7185 raft_consensus.cc:1261] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 84bcfcb7bf89438591ec47d68d7fdd2d [term 1 FOLLOWER]: Refusing update from remote peer 43bd0dd954344685be35f76ee0857d2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:22.152925 7397 raft_consensus.cc:1261] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 709ada1eccfe441397baf0e242e447ce [term 1 FOLLOWER]: Refusing update from remote peer 43bd0dd954344685be35f76ee0857d2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:22.153229 7707 consensus_queue.cc:1035] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "84bcfcb7bf89438591ec47d68d7fdd2d" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:22.153357 7713 consensus_queue.cc:1035] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "709ada1eccfe441397baf0e242e447ce" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:22.155575 7717 mvcc.cc:205] Tried to move back new op lower bound from 7315573154416005120 to 7315573154330476544. Current Snapshot: MvccSnapshot[applied={T|T < 7315573154416005120}]
I20260806 15:03:22.155684 7718 mvcc.cc:205] Tried to move back new op lower bound from 7315573154416005120 to 7315573154330476544. Current Snapshot: MvccSnapshot[applied={T|T < 7315573154416005120}]
I20260806 15:03:22.155766 7719 mvcc.cc:205] Tried to move back new op lower bound from 7315573154416005120 to 7315573154330476544. Current Snapshot: MvccSnapshot[applied={T|T < 7315573154416005120}]
I20260806 15:03:22.158159 6965 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:22.158258 6965 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:22.158953 6965 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c9741a25582540c79d33470f8d7b3cfa: Sending DeleteTablet for 3 replicas of tablet 4d04e04c39e648e0a8ef4ea4ef74dcb4
I20260806 15:03:22.159271 7154 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d04e04c39e648e0a8ef4ea4ef74dcb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:22 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:33884
I20260806 15:03:22.159302 7578 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d04e04c39e648e0a8ef4ea4ef74dcb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:22 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:58472
I20260806 15:03:22.159507 7371 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d04e04c39e648e0a8ef4ea4ef74dcb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:22 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:34748
I20260806 15:03:22.159518 7733 tablet_replica.cc:330] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c: stopping tablet replica
I20260806 15:03:22.159611 7733 raft_consensus.cc:2229] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:03:22.159685 7734 tablet_replica.cc:330] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 709ada1eccfe441397baf0e242e447ce: stopping tablet replica
I20260806 15:03:22.159713 7733 raft_consensus.cc:2258] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:22.159766 7734 raft_consensus.cc:2229] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 709ada1eccfe441397baf0e242e447ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:22.159853 7734 raft_consensus.cc:2258] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 709ada1eccfe441397baf0e242e447ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:22.160008 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 7062
I20260806 15:03:22.160360 7734 ts_tablet_manager.cc:1905] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 709ada1eccfe441397baf0e242e447ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:22.161684 7734 ts_tablet_manager.cc:1918] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 709ada1eccfe441397baf0e242e447ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:03:22.161751 7734 log.cc:1196] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 709ada1eccfe441397baf0e242e447ce: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-1/wal/wals/4d04e04c39e648e0a8ef4ea4ef74dcb4
I20260806 15:03:22.162029 7734 ts_tablet_manager.cc:1939] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 709ada1eccfe441397baf0e242e447ce: Deleting consensus metadata
I20260806 15:03:22.162372 6949 catalog_manager.cc:4841] TS 709ada1eccfe441397baf0e242e447ce (127.0.236.66:42709): tablet 4d04e04c39e648e0a8ef4ea4ef74dcb4 (table test-table [id=7ae725599b9a4627a53e011f560e68a6]) successfully deleted
I20260806 15:03:22.174258 7733 ts_tablet_manager.cc:1905] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:22.175879 7733 ts_tablet_manager.cc:1918] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:03:22.175972 7733 log.cc:1196] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/4d04e04c39e648e0a8ef4ea4ef74dcb4
I20260806 15:03:22.176196 7733 ts_tablet_manager.cc:1939] T 4d04e04c39e648e0a8ef4ea4ef74dcb4 P 43bd0dd954344685be35f76ee0857d2c: Deleting consensus metadata
I20260806 15:03:22.176597 6947 catalog_manager.cc:4841] TS 43bd0dd954344685be35f76ee0857d2c (127.0.236.67:38443): tablet 4d04e04c39e648e0a8ef4ea4ef74dcb4 (table test-table [id=7ae725599b9a4627a53e011f560e68a6]) successfully deleted
W20260806 15:03:22.179255 6949 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.236.65:45979 (error 108)
I20260806 15:03:22.179528 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 7268
I20260806 15:03:22.181012 7684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.179612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:45979 (local address 127.0.0.1:33898)
0806 15:03:22.180925 (+ 1313us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:22.180974 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.65:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1290}
W20260806 15:03:22.181473 6949 catalog_manager.cc:4584] TS 84bcfcb7bf89438591ec47d68d7fdd2d (127.0.236.65:45979): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4d04e04c39e648e0a8ef4ea4ef74dcb4: Network error: Client connection negotiation failed: client connection to 127.0.236.65:45979: connect: Connection refused (error 111)
I20260806 15:03:22.187988 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 7477
I20260806 15:03:22.198693 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 6927
2026-08-06T15:03:22Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1990 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T15:03:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:03:22Z Disabled control of system clock
I20260806 15:03:22.227824 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:43671
--webserver_interface=127.0.236.126
--webserver_port=0
--builtin_ntp_servers=127.0.236.84:36845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:43671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 15:03:22.306571 7743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:22.306767 7743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:22.306788 7743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:22.307870 7743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:03:22.307906 7743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:22.307919 7743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:22.307930 7743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:03:22.307956 7743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:03:22.309237 7743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:36845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:43671
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:43671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:22.309419 7743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:22.309633 7743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:22.314075 7751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:22.314085 7753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:22.314111 7743 server_base.cc:1034] running on GCE node
W20260806 15:03:22.314085 7749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:22.314579 7743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:22.314842 7743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:22.315979 7743 hybrid_clock.cc:648] HybridClock initialized: now 1786028602315971 us; error 34 us; skew 500 ppm
I20260806 15:03:22.317060 7743 webserver.cc:460] Webserver started at http://127.0.236.126:36059/ using document root <none> and password file <none>
I20260806 15:03:22.317260 7743 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:22.317302 7743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:22.317425 7743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:22.318429 7743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "cde00388a34a405abcf7262e0d8d0fed"
format_stamp: "Formatted at 2026-08-06 15:03:22 on dist-test-slave-tc2s"
server_key: "e0a10da3fe7412331bcbbb353b7c631d"
server_key_iv: "556d1e0f070582a67b55fc2dbf46f9b8"
server_key_version: "encryptionkey@0"
I20260806 15:03:22.318753 7743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "cde00388a34a405abcf7262e0d8d0fed"
format_stamp: "Formatted at 2026-08-06 15:03:22 on dist-test-slave-tc2s"
server_key: "e0a10da3fe7412331bcbbb353b7c631d"
server_key_iv: "556d1e0f070582a67b55fc2dbf46f9b8"
server_key_version: "encryptionkey@0"
I20260806 15:03:22.319981 7743 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:03:22.320607 7762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:22.320756 7743 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:22.320844 7743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "cde00388a34a405abcf7262e0d8d0fed"
format_stamp: "Formatted at 2026-08-06 15:03:22 on dist-test-slave-tc2s"
server_key: "e0a10da3fe7412331bcbbb353b7c631d"
server_key_iv: "556d1e0f070582a67b55fc2dbf46f9b8"
server_key_version: "encryptionkey@0"
I20260806 15:03:22.320906 7743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:22.324020 7743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:22.324256 7743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:22.324374 7743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:22.328168 7743 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:43671
I20260806 15:03:22.328200 7842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:43671 every 8 connection(s)
I20260806 15:03:22.328487 7743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:03:22.329097 7843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:22.331446 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 7743
I20260806 15:03:22.331524 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:03:22.331669 945 external_mini_cluster.cc:1428] Setting key ca8b2789d45e381931e1911f11564937
I20260806 15:03:22.331868 7843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cde00388a34a405abcf7262e0d8d0fed: Bootstrap starting.
I20260806 15:03:22.332574 7843 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cde00388a34a405abcf7262e0d8d0fed: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:22.332894 7843 log.cc:824] T 00000000000000000000000000000000 P cde00388a34a405abcf7262e0d8d0fed: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:22.333606 7843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cde00388a34a405abcf7262e0d8d0fed: No bootstrap required, opened a new log
I20260806 15:03:22.334215 7846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.332124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34236 (local address 127.0.236.126:43671)
0806 15:03:22.332402 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:22.332410 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:22.332439 (+ 29us) server_negotiation.cc:407] Connection header received
0806 15:03:22.332969 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:22.332978 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:22.333028 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:22.333169 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:22.333778 (+ 609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:22.333782 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:22.333782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:03:22.333787 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:22.333833 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:03:22.333843 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:22.333929 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:22.334010 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 15:03:22.334065 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":64,"threads_started":1}
I20260806 15:03:22.335131 7843 raft_consensus.cc:348] T 00000000000000000000000000000000 P cde00388a34a405abcf7262e0d8d0fed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cde00388a34a405abcf7262e0d8d0fed" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43671 } }
I20260806 15:03:22.335254 7843 raft_consensus.cc:374] T 00000000000000000000000000000000 P cde00388a34a405abcf7262e0d8d0fed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:22.335287 7843 raft_consensus.cc:729] T 00000000000000000000000000000000 P cde00388a34a405abcf7262e0d8d0fed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cde00388a34a405abcf7262e0d8d0fed, State: Initialized, Role: FOLLOWER
I20260806 15:03:22.335397 7843 consensus_queue.cc:260] T 00000000000000000000000000000000 P cde00388a34a405abcf7262e0d8d0fed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cde00388a34a405abcf7262e0d8d0fed" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43671 } }
I20260806 15:03:22.335505 7843 raft_consensus.cc:388] T 00000000000000000000000000000000 P cde00388a34a405abcf7262e0d8d0fed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:03:22.335548 7843 raft_consensus.cc:482] T 00000000000000000000000000000000 P cde00388a34a405abcf7262e0d8d0fed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:03:22.335595 7843 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cde00388a34a405abcf7262e0d8d0fed [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:22.336314 7843 raft_consensus.cc:504] T 00000000000000000000000000000000 P cde00388a34a405abcf7262e0d8d0fed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cde00388a34a405abcf7262e0d8d0fed" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43671 } }
I20260806 15:03:22.336406 7843 leader_election.cc:302] T 00000000000000000000000000000000 P cde00388a34a405abcf7262e0d8d0fed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cde00388a34a405abcf7262e0d8d0fed; no voters:
I20260806 15:03:22.336561 7843 leader_election.cc:288] T 00000000000000000000000000000000 P cde00388a34a405abcf7262e0d8d0fed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:03:22.336606 7849 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cde00388a34a405abcf7262e0d8d0fed [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:22.336736 7849 raft_consensus.cc:686] T 00000000000000000000000000000000 P cde00388a34a405abcf7262e0d8d0fed [term 1 LEADER]: Becoming Leader. State: Replica: cde00388a34a405abcf7262e0d8d0fed, State: Running, Role: LEADER
I20260806 15:03:22.336803 7843 sys_catalog.cc:564] T 00000000000000000000000000000000 P cde00388a34a405abcf7262e0d8d0fed [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:03:22.336838 7849 consensus_queue.cc:237] T 00000000000000000000000000000000 P cde00388a34a405abcf7262e0d8d0fed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cde00388a34a405abcf7262e0d8d0fed" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43671 } }
I20260806 15:03:22.337210 7850 sys_catalog.cc:455] T 00000000000000000000000000000000 P cde00388a34a405abcf7262e0d8d0fed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cde00388a34a405abcf7262e0d8d0fed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cde00388a34a405abcf7262e0d8d0fed" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43671 } } }
I20260806 15:03:22.337361 7850 sys_catalog.cc:458] T 00000000000000000000000000000000 P cde00388a34a405abcf7262e0d8d0fed [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:22.337579 7861 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:03:22.337208 7851 sys_catalog.cc:455] T 00000000000000000000000000000000 P cde00388a34a405abcf7262e0d8d0fed [sys.catalog]: SysCatalogTable state changed. Reason: New leader cde00388a34a405abcf7262e0d8d0fed. Latest consensus state: current_term: 1 leader_uuid: "cde00388a34a405abcf7262e0d8d0fed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cde00388a34a405abcf7262e0d8d0fed" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43671 } } }
I20260806 15:03:22.337968 7851 sys_catalog.cc:458] T 00000000000000000000000000000000 P cde00388a34a405abcf7262e0d8d0fed [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:22.342057 7861 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:03:22.343551 7861 catalog_manager.cc:1347] Generated new cluster ID: 79b07d856ec54859bf4e6f61c819f691
I20260806 15:03:22.343603 7861 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:03:22.353677 7861 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:03:22.353822 7861 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:03:22.366482 7861 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cde00388a34a405abcf7262e0d8d0fed: Generated new TSK 0
I20260806 15:03:22.366640 7861 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:03:22.373334 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:0
--local_ip_for_outbound_sockets=127.0.236.65
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:43671
--builtin_ntp_servers=127.0.236.84:36845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 15:03:22.453227 7877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:22.453435 7877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:22.453467 7877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:22.454660 7877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:22.454703 7877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:22.454736 7877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:03:22.456064 7877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:36845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:43671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:22.456276 7877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:22.456532 7877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:22.462061 7889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:22.462069 7892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:22.462116 7877 server_base.cc:1034] running on GCE node
W20260806 15:03:22.462069 7885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:22.462460 7877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:22.462677 7877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:22.463811 7877 hybrid_clock.cc:648] HybridClock initialized: now 1786028602463800 us; error 29 us; skew 500 ppm
I20260806 15:03:22.464972 7877 webserver.cc:460] Webserver started at http://127.0.236.65:39273/ using document root <none> and password file <none>
I20260806 15:03:22.465173 7877 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:22.465214 7877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:22.465334 7877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:22.466363 7877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-0/data/instance:
uuid: "4124281d1ce84becb16b06adfbd47976"
format_stamp: "Formatted at 2026-08-06 15:03:22 on dist-test-slave-tc2s"
server_key: "482a0815ffa2364a66e882a9485a29aa"
server_key_iv: "2411990b17f29d956db97e7c712da3d9"
server_key_version: "encryptionkey@0"
I20260806 15:03:22.466678 7877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-0/wal/instance:
uuid: "4124281d1ce84becb16b06adfbd47976"
format_stamp: "Formatted at 2026-08-06 15:03:22 on dist-test-slave-tc2s"
server_key: "482a0815ffa2364a66e882a9485a29aa"
server_key_iv: "2411990b17f29d956db97e7c712da3d9"
server_key_version: "encryptionkey@0"
I20260806 15:03:22.467831 7877 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:03:22.468586 7903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:22.468729 7877 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:22.468813 7877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "4124281d1ce84becb16b06adfbd47976"
format_stamp: "Formatted at 2026-08-06 15:03:22 on dist-test-slave-tc2s"
server_key: "482a0815ffa2364a66e882a9485a29aa"
server_key_iv: "2411990b17f29d956db97e7c712da3d9"
server_key_version: "encryptionkey@0"
I20260806 15:03:22.468894 7877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:22.472103 7877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:22.472325 7877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:22.472436 7877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:22.472621 7877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:22.472903 7877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:22.472935 7877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:22.472956 7877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:22.472978 7877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:22.478219 7877 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:45143
I20260806 15:03:22.478271 8081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:45143 every 8 connection(s)
I20260806 15:03:22.478626 7877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:03:22.481343 7846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.479289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:41115 (local address 127.0.236.126:43671)
0806 15:03:22.479338 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:22.479340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:22.479567 (+ 227us) server_negotiation.cc:407] Connection header received
0806 15:03:22.479991 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:22.479992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:22.479998 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:22.480032 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:22.481167 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:22.481168 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:22.481168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:03:22.481172 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:22.481187 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:03:22.481190 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:22.481233 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:22.481279 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 15:03:22.481291 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:03:22.481494 8085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.479272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43671 (local address 127.0.236.65:41115)
0806 15:03:22.479460 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:22.479492 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:22.479905 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:22.480087 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:22.480090 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:22.480098 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:22.481037 (+ 939us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:03:22.481038 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:22.481066 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:03:22.481092 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:22.481233 (+ 141us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:22.481235 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:22.481237 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 15:03:22.481280 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 15:03:22.481324 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":87,"threads_started":1}
I20260806 15:03:22.481966 8082 heartbeater.cc:347] Connected to a master server at 127.0.236.126:43671
I20260806 15:03:22.482056 8082 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:22.482193 8082 heartbeater.cc:511] Master 127.0.236.126:43671 requested a full tablet report, sending...
I20260806 15:03:22.482677 7791 ts_manager.cc:194] Registered new tserver with Master: 4124281d1ce84becb16b06adfbd47976 (127.0.236.65:45143)
I20260806 15:03:22.488508 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 7877
I20260806 15:03:22.488600 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-0/wal/instance
I20260806 15:03:22.488768 945 external_mini_cluster.cc:1428] Setting key 6200223fd5881c604cc2a88362700380
I20260806 15:03:22.489598 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:0
--local_ip_for_outbound_sockets=127.0.236.66
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:43671
--builtin_ntp_servers=127.0.236.84:36845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 15:03:22.569033 8086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:22.569216 8086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:22.569244 8086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:22.570379 8086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:22.570425 8086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:22.570459 8086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:03:22.571833 8086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:36845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:43671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:22.572059 8086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:22.572337 8086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:22.576978 8094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:22.576944 8095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:22.577023 8086 server_base.cc:1034] running on GCE node
W20260806 15:03:22.576963 8097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:22.577344 8086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:22.577603 8086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:22.578735 8086 hybrid_clock.cc:648] HybridClock initialized: now 1786028602578721 us; error 29 us; skew 500 ppm
I20260806 15:03:22.579878 8086 webserver.cc:460] Webserver started at http://127.0.236.66:34263/ using document root <none> and password file <none>
I20260806 15:03:22.580108 8086 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:22.580157 8086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:22.580282 8086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:22.581256 8086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-1/data/instance:
uuid: "23cab921e5cb4b44b028840f644cea30"
format_stamp: "Formatted at 2026-08-06 15:03:22 on dist-test-slave-tc2s"
server_key: "10ee58d95c1307e483cb560d9d38df6f"
server_key_iv: "3745ecbbf6075f9e080cdf93e2c879e8"
server_key_version: "encryptionkey@0"
I20260806 15:03:22.581573 8086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-1/wal/instance:
uuid: "23cab921e5cb4b44b028840f644cea30"
format_stamp: "Formatted at 2026-08-06 15:03:22 on dist-test-slave-tc2s"
server_key: "10ee58d95c1307e483cb560d9d38df6f"
server_key_iv: "3745ecbbf6075f9e080cdf93e2c879e8"
server_key_version: "encryptionkey@0"
I20260806 15:03:22.582798 8086 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:03:22.583422 8104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:22.583575 8086 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:22.583655 8086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "23cab921e5cb4b44b028840f644cea30"
format_stamp: "Formatted at 2026-08-06 15:03:22 on dist-test-slave-tc2s"
server_key: "10ee58d95c1307e483cb560d9d38df6f"
server_key_iv: "3745ecbbf6075f9e080cdf93e2c879e8"
server_key_version: "encryptionkey@0"
I20260806 15:03:22.583719 8086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:22.586879 8086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:22.587106 8086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:22.587218 8086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:22.587399 8086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:22.587666 8086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:22.587698 8086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:22.587724 8086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:22.587746 8086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:22.592974 8086 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:37447
I20260806 15:03:22.593034 8262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:37447 every 8 connection(s)
I20260806 15:03:22.593283 8086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
I20260806 15:03:22.594231 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 8086
I20260806 15:03:22.594339 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-1/wal/instance
I20260806 15:03:22.594513 945 external_mini_cluster.cc:1428] Setting key 3ac472f376392dcea9e17c27b712f545
I20260806 15:03:22.595420 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:0
--local_ip_for_outbound_sockets=127.0.236.67
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:43671
--builtin_ntp_servers=127.0.236.84:36845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 15:03:22.596113 7846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.593881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:54227 (local address 127.0.236.126:43671)
0806 15:03:22.593947 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:22.593949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:22.594194 (+ 245us) server_negotiation.cc:407] Connection header received
0806 15:03:22.594749 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:22.594751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:22.594758 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:22.594821 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:22.595883 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:22.595885 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:22.595885 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:03:22.595890 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:22.595908 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:03:22.595913 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:22.595987 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:22.596031 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 15:03:22.596044 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 15:03:22.596251 8265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.593865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43671 (local address 127.0.236.66:54227)
0806 15:03:22.594059 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:22.594099 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:22.594645 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:22.594868 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:22.594871 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:22.594879 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:22.595743 (+ 864us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:03:22.595744 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:22.595770 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:03:22.595795 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:22.595985 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:22.595987 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:22.595989 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 15:03:22.596036 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 15:03:22.596080 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":95,"threads_started":1}
I20260806 15:03:22.597224 8263 heartbeater.cc:347] Connected to a master server at 127.0.236.126:43671
I20260806 15:03:22.597302 8263 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:22.597433 8263 heartbeater.cc:511] Master 127.0.236.126:43671 requested a full tablet report, sending...
I20260806 15:03:22.597771 7791 ts_manager.cc:194] Registered new tserver with Master: 23cab921e5cb4b44b028840f644cea30 (127.0.236.66:37447)
W20260806 15:03:22.675710 8266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:22.675894 8266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:22.675930 8266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:22.677052 8266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:22.677089 8266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:22.677124 8266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:03:22.678448 8266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:36845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:43671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:22.678661 8266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:22.678908 8266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:22.683449 8279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:22.683547 8276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:22.683499 8274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:22.683791 8266 server_base.cc:1034] running on GCE node
I20260806 15:03:22.683974 8266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:22.684180 8266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:22.685315 8266 hybrid_clock.cc:648] HybridClock initialized: now 1786028602685305 us; error 23 us; skew 500 ppm
I20260806 15:03:22.686530 8266 webserver.cc:460] Webserver started at http://127.0.236.67:40889/ using document root <none> and password file <none>
I20260806 15:03:22.686728 8266 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:22.686774 8266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:22.686895 8266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:22.687913 8266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-2/data/instance:
uuid: "8494a0194ec54e729ed1c8d22233b1e0"
format_stamp: "Formatted at 2026-08-06 15:03:22 on dist-test-slave-tc2s"
server_key: "9970c5de9a71052b0982955938662ecf"
server_key_iv: "3d1ae770ed5df7f8501b54b8d91ed5c0"
server_key_version: "encryptionkey@0"
I20260806 15:03:22.688230 8266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-2/wal/instance:
uuid: "8494a0194ec54e729ed1c8d22233b1e0"
format_stamp: "Formatted at 2026-08-06 15:03:22 on dist-test-slave-tc2s"
server_key: "9970c5de9a71052b0982955938662ecf"
server_key_iv: "3d1ae770ed5df7f8501b54b8d91ed5c0"
server_key_version: "encryptionkey@0"
I20260806 15:03:22.689380 8266 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:03:22.690028 8286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:22.690169 8266 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:03:22.690250 8266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "8494a0194ec54e729ed1c8d22233b1e0"
format_stamp: "Formatted at 2026-08-06 15:03:22 on dist-test-slave-tc2s"
server_key: "9970c5de9a71052b0982955938662ecf"
server_key_iv: "3d1ae770ed5df7f8501b54b8d91ed5c0"
server_key_version: "encryptionkey@0"
I20260806 15:03:22.690315 8266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:22.693373 8266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:22.693607 8266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:22.693719 8266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:22.693931 8266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:22.694221 8266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:22.694253 8266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:22.694279 8266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:22.694296 8266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:22.699462 8266 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:40687
I20260806 15:03:22.699510 8454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:40687 every 8 connection(s)
I20260806 15:03:22.699779 8266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
I20260806 15:03:22.700455 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 8266
I20260806 15:03:22.700547 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-2/wal/instance
I20260806 15:03:22.700701 945 external_mini_cluster.cc:1428] Setting key b35aeff4b05b2f0123a8bf73124c04e5
I20260806 15:03:22.702442 7846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.700374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:47241 (local address 127.0.236.126:43671)
0806 15:03:22.700430 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:22.700433 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:22.700633 (+ 200us) server_negotiation.cc:407] Connection header received
0806 15:03:22.701155 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:22.701156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:22.701162 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:22.701203 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:22.702267 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:22.702268 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:22.702269 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:03:22.702273 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:22.702287 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:03:22.702291 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:22.702335 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:22.702379 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 15:03:22.702390 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 15:03:22.702656 8457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.700354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43671 (local address 127.0.236.67:47241)
0806 15:03:22.700528 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:22.700564 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:22.701053 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:22.701275 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:22.701279 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:22.701289 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:22.702144 (+ 855us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:03:22.702145 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:22.702169 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:03:22.702194 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:22.702334 (+ 140us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:22.702336 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:22.702339 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:03:22.702384 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 15:03:22.702427 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":89,"threads_started":1}
I20260806 15:03:22.703058 8455 heartbeater.cc:347] Connected to a master server at 127.0.236.126:43671
I20260806 15:03:22.703135 8455 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:22.703264 8455 heartbeater.cc:511] Master 127.0.236.126:43671 requested a full tablet report, sending...
I20260806 15:03:22.703534 7791 ts_manager.cc:194] Registered new tserver with Master: 8494a0194ec54e729ed1c8d22233b1e0 (127.0.236.67:40687)
I20260806 15:03:22.712181 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:03:22.714190 7846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.713354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42024 (local address 127.0.236.126:43671)
0806 15:03:22.713397 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:22.713398 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:22.713494 (+ 96us) server_negotiation.cc:407] Connection header received
0806 15:03:22.713533 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:22.713533 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:22.713537 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:22.713582 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:22.714037 (+ 455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:22.714038 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:22.714038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:03:22.714041 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:22.714053 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:03:22.714056 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:22.714097 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:22.714135 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 15:03:22.714144 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 15:03:22.715616 7791 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:03:22.717101 7791 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:03:22.720741 8468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.719027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47534 (local address 127.0.236.67:40687)
0806 15:03:22.719177 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:22.719182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:22.719319 (+ 137us) server_negotiation.cc:407] Connection header received
0806 15:03:22.719368 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:22.719372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:22.719381 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:22.719431 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:22.720491 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:22.720499 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:22.720499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:03:22.720504 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:22.720539 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:03:22.720561 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:22.720616 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:22.720661 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 15:03:22.720674 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
I20260806 15:03:22.720742 8465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.718975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:40687 (local address 127.0.0.1:47534)
0806 15:03:22.719221 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:22.719256 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:22.719311 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:22.719726 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:22.719727 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:22.719730 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:22.720410 (+ 680us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:03:22.720410 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:22.720416 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:03:22.720426 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:22.720613 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:22.720614 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:22.720615 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 15:03:22.720656 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 15:03:22.720668 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":60,"threads_started":1}
I20260806 15:03:22.720808 8471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.719400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46082 (local address 127.0.236.65:45143)
0806 15:03:22.719557 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:22.719561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:22.719571 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:03:22.719597 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:22.719599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:22.719607 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:22.719690 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:22.720364 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:22.720372 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:22.720372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:03:22.720376 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:22.720412 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:03:22.720445 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:22.720503 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:22.720714 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 15:03:22.720730 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
I20260806 15:03:22.720655 8467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.718984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:45143 (local address 127.0.0.1:46082)
0806 15:03:22.719228 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:22.719262 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:22.719314 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:22.719699 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:22.719704 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:22.719709 (+ 5us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:22.720236 (+ 527us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:03:22.720236 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:22.720251 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:03:22.720273 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:22.720504 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:22.720510 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:22.720513 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:03:22.720577 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 15:03:22.720589 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":85,"threads_started":1}
I20260806 15:03:22.721316 8361 tablet_service.cc:1467] Processing CreateTablet for tablet 1ecb4b89b655486180007ab88ceae805 (DEFAULT_TABLE table=test-table [id=3b84de903d1c4e99ab9a991cea209dca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:22.721531 7977 tablet_service.cc:1467] Processing CreateTablet for tablet 1ecb4b89b655486180007ab88ceae805 (DEFAULT_TABLE table=test-table [id=3b84de903d1c4e99ab9a991cea209dca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:22.721688 8361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ecb4b89b655486180007ab88ceae805. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:22.721859 7977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ecb4b89b655486180007ab88ceae805. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:22.722093 8470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.719207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35908 (local address 127.0.236.66:37447)
0806 15:03:22.719411 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:22.719415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:22.720344 (+ 929us) server_negotiation.cc:407] Connection header received
0806 15:03:22.720569 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:22.720574 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:22.720584 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:22.720654 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:22.721756 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:22.721763 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:22.721764 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:03:22.721769 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:22.721803 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:03:22.721830 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:22.721861 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:22.721998 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 15:03:22.722015 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":89,"threads_started":1}
I20260806 15:03:22.722272 8464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.718987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:37447 (local address 127.0.0.1:35908)
0806 15:03:22.720271 (+ 1284us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:22.720280 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:22.720370 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:22.721275 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:22.721276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:22.721278 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:22.721645 (+ 367us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:03:22.721645 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:22.721652 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:03:22.721664 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:22.721922 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:22.721923 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:22.721924 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 15:03:22.722201 (+ 277us) client_negotiation.cc:240] Negotiation successful
0806 15:03:22.722214 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1271,"thread_start_us":72,"threads_started":1}
I20260806 15:03:22.722824 8170 tablet_service.cc:1467] Processing CreateTablet for tablet 1ecb4b89b655486180007ab88ceae805 (DEFAULT_TABLE table=test-table [id=3b84de903d1c4e99ab9a991cea209dca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:22.723155 8170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ecb4b89b655486180007ab88ceae805. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:22.724956 8474 tablet_bootstrap.cc:492] T 1ecb4b89b655486180007ab88ceae805 P 8494a0194ec54e729ed1c8d22233b1e0: Bootstrap starting.
I20260806 15:03:22.725414 8475 tablet_bootstrap.cc:492] T 1ecb4b89b655486180007ab88ceae805 P 4124281d1ce84becb16b06adfbd47976: Bootstrap starting.
I20260806 15:03:22.725772 8478 tablet_bootstrap.cc:492] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30: Bootstrap starting.
I20260806 15:03:22.725816 8474 tablet_bootstrap.cc:654] T 1ecb4b89b655486180007ab88ceae805 P 8494a0194ec54e729ed1c8d22233b1e0: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:22.726054 8475 tablet_bootstrap.cc:654] T 1ecb4b89b655486180007ab88ceae805 P 4124281d1ce84becb16b06adfbd47976: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:22.726125 8474 log.cc:824] T 1ecb4b89b655486180007ab88ceae805 P 8494a0194ec54e729ed1c8d22233b1e0: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:22.726364 8475 log.cc:824] T 1ecb4b89b655486180007ab88ceae805 P 4124281d1ce84becb16b06adfbd47976: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:22.726416 8478 tablet_bootstrap.cc:654] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:22.726696 8478 log.cc:824] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:22.726830 8474 tablet_bootstrap.cc:492] T 1ecb4b89b655486180007ab88ceae805 P 8494a0194ec54e729ed1c8d22233b1e0: No bootstrap required, opened a new log
I20260806 15:03:22.726894 8474 ts_tablet_manager.cc:1397] T 1ecb4b89b655486180007ab88ceae805 P 8494a0194ec54e729ed1c8d22233b1e0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260806 15:03:22.727082 8475 tablet_bootstrap.cc:492] T 1ecb4b89b655486180007ab88ceae805 P 4124281d1ce84becb16b06adfbd47976: No bootstrap required, opened a new log
I20260806 15:03:22.727152 8475 ts_tablet_manager.cc:1397] T 1ecb4b89b655486180007ab88ceae805 P 4124281d1ce84becb16b06adfbd47976: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:03:22.727375 8478 tablet_bootstrap.cc:492] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30: No bootstrap required, opened a new log
I20260806 15:03:22.727488 8478 ts_tablet_manager.cc:1397] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:03:22.729357 8478 raft_consensus.cc:348] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8494a0194ec54e729ed1c8d22233b1e0" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 40687 } } peers { permanent_uuid: "4124281d1ce84becb16b06adfbd47976" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45143 } } peers { permanent_uuid: "23cab921e5cb4b44b028840f644cea30" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37447 } }
I20260806 15:03:22.729333 8474 raft_consensus.cc:348] T 1ecb4b89b655486180007ab88ceae805 P 8494a0194ec54e729ed1c8d22233b1e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8494a0194ec54e729ed1c8d22233b1e0" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 40687 } } peers { permanent_uuid: "4124281d1ce84becb16b06adfbd47976" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45143 } } peers { permanent_uuid: "23cab921e5cb4b44b028840f644cea30" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37447 } }
I20260806 15:03:22.729477 8478 raft_consensus.cc:374] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:22.729494 8474 raft_consensus.cc:374] T 1ecb4b89b655486180007ab88ceae805 P 8494a0194ec54e729ed1c8d22233b1e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:22.729508 8478 raft_consensus.cc:729] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cab921e5cb4b44b028840f644cea30, State: Initialized, Role: FOLLOWER
I20260806 15:03:22.729543 8474 raft_consensus.cc:729] T 1ecb4b89b655486180007ab88ceae805 P 8494a0194ec54e729ed1c8d22233b1e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8494a0194ec54e729ed1c8d22233b1e0, State: Initialized, Role: FOLLOWER
I20260806 15:03:22.729498 8475 raft_consensus.cc:348] T 1ecb4b89b655486180007ab88ceae805 P 4124281d1ce84becb16b06adfbd47976 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8494a0194ec54e729ed1c8d22233b1e0" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 40687 } } peers { permanent_uuid: "4124281d1ce84becb16b06adfbd47976" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45143 } } peers { permanent_uuid: "23cab921e5cb4b44b028840f644cea30" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37447 } }
I20260806 15:03:22.729625 8475 raft_consensus.cc:374] T 1ecb4b89b655486180007ab88ceae805 P 4124281d1ce84becb16b06adfbd47976 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:22.729663 8475 raft_consensus.cc:729] T 1ecb4b89b655486180007ab88ceae805 P 4124281d1ce84becb16b06adfbd47976 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4124281d1ce84becb16b06adfbd47976, State: Initialized, Role: FOLLOWER
I20260806 15:03:22.729643 8478 consensus_queue.cc:260] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8494a0194ec54e729ed1c8d22233b1e0" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 40687 } } peers { permanent_uuid: "4124281d1ce84becb16b06adfbd47976" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45143 } } peers { permanent_uuid: "23cab921e5cb4b44b028840f644cea30" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37447 } }
I20260806 15:03:22.729674 8474 consensus_queue.cc:260] T 1ecb4b89b655486180007ab88ceae805 P 8494a0194ec54e729ed1c8d22233b1e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8494a0194ec54e729ed1c8d22233b1e0" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 40687 } } peers { permanent_uuid: "4124281d1ce84becb16b06adfbd47976" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45143 } } peers { permanent_uuid: "23cab921e5cb4b44b028840f644cea30" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37447 } }
I20260806 15:03:22.729785 8475 consensus_queue.cc:260] T 1ecb4b89b655486180007ab88ceae805 P 4124281d1ce84becb16b06adfbd47976 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8494a0194ec54e729ed1c8d22233b1e0" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 40687 } } peers { permanent_uuid: "4124281d1ce84becb16b06adfbd47976" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45143 } } peers { permanent_uuid: "23cab921e5cb4b44b028840f644cea30" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37447 } }
I20260806 15:03:22.729885 8478 ts_tablet_manager.cc:1428] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:03:22.729941 8474 ts_tablet_manager.cc:1428] T 1ecb4b89b655486180007ab88ceae805 P 8494a0194ec54e729ed1c8d22233b1e0: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 15:03:22.730019 8263 heartbeater.cc:503] Master 127.0.236.126:43671 was elected leader, sending a full tablet report...
I20260806 15:03:22.730044 8475 ts_tablet_manager.cc:1428] T 1ecb4b89b655486180007ab88ceae805 P 4124281d1ce84becb16b06adfbd47976: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 15:03:22.730086 8455 heartbeater.cc:503] Master 127.0.236.126:43671 was elected leader, sending a full tablet report...
I20260806 15:03:22.730309 8082 heartbeater.cc:503] Master 127.0.236.126:43671 was elected leader, sending a full tablet report...
I20260806 15:03:22.834589 8482 raft_consensus.cc:482] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:03:22.834715 8482 raft_consensus.cc:504] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8494a0194ec54e729ed1c8d22233b1e0" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 40687 } } peers { permanent_uuid: "4124281d1ce84becb16b06adfbd47976" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45143 } } peers { permanent_uuid: "23cab921e5cb4b44b028840f644cea30" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37447 } }
I20260806 15:03:22.835041 8482 leader_election.cc:288] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8494a0194ec54e729ed1c8d22233b1e0 (127.0.236.67:40687), 4124281d1ce84becb16b06adfbd47976 (127.0.236.65:45143)
I20260806 15:03:22.836328 8265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.835198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:40687 (local address 127.0.236.66:40163)
0806 15:03:22.835250 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:22.835262 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:22.835307 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:22.835555 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:22.835556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:22.835559 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:22.836022 (+ 463us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:03:22.836023 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:22.836031 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:03:22.836045 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:22.836207 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:22.836209 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:22.836209 (+ 0us) client_negotiation.cc:769] Sending connection context
0806 15:03:22.836251 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 15:03:22.836264 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 15:03:22.836339 8468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.835362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:40163 (local address 127.0.236.67:40687)
0806 15:03:22.835403 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:22.835405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:22.835425 (+ 20us) server_negotiation.cc:407] Connection header received
0806 15:03:22.835446 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:22.835448 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:22.835453 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:22.835500 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:22.836106 (+ 606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:22.836107 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:22.836107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:03:22.836111 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:22.836128 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:03:22.836132 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:22.836188 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:22.836262 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 15:03:22.836276 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:03:22.836659 8486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.835198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:45143 (local address 127.0.236.66:42591)
0806 15:03:22.835397 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:22.835405 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:22.835453 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:22.835616 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:22.835617 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:22.835619 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:22.836203 (+ 584us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:03:22.836204 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:22.836211 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:03:22.836223 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:22.836518 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:22.836519 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:22.836520 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 15:03:22.836573 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 15:03:22.836586 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":66,"thread_start_us":81,"threads_started":1}
I20260806 15:03:22.836660 8471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.835279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:42591 (local address 127.0.236.65:45143)
0806 15:03:22.835336 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:22.835338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:22.835461 (+ 123us) server_negotiation.cc:407] Connection header received
0806 15:03:22.835523 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:22.835524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:22.835530 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:22.835585 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:22.836291 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:22.836292 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:22.836293 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:03:22.836297 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:22.836313 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:03:22.836317 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:22.836385 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:22.836585 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 15:03:22.836599 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 15:03:22.836644 8388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ecb4b89b655486180007ab88ceae805" candidate_uuid: "23cab921e5cb4b44b028840f644cea30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8494a0194ec54e729ed1c8d22233b1e0" is_pre_election: true
I20260806 15:03:22.836784 8388 raft_consensus.cc:2454] T 1ecb4b89b655486180007ab88ceae805 P 8494a0194ec54e729ed1c8d22233b1e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23cab921e5cb4b44b028840f644cea30 in term 0.
I20260806 15:03:22.837003 8110 leader_election.cc:302] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cab921e5cb4b44b028840f644cea30, 8494a0194ec54e729ed1c8d22233b1e0; no voters:
I20260806 15:03:22.837121 8482 raft_consensus.cc:2781] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:03:22.837109 8010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ecb4b89b655486180007ab88ceae805" candidate_uuid: "23cab921e5cb4b44b028840f644cea30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124281d1ce84becb16b06adfbd47976" is_pre_election: true
I20260806 15:03:22.837175 8482 raft_consensus.cc:482] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:03:22.837191 8482 raft_consensus.cc:3037] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:22.837240 8010 raft_consensus.cc:2454] T 1ecb4b89b655486180007ab88ceae805 P 4124281d1ce84becb16b06adfbd47976 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23cab921e5cb4b44b028840f644cea30 in term 0.
I20260806 15:03:22.838056 8482 raft_consensus.cc:504] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8494a0194ec54e729ed1c8d22233b1e0" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 40687 } } peers { permanent_uuid: "4124281d1ce84becb16b06adfbd47976" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45143 } } peers { permanent_uuid: "23cab921e5cb4b44b028840f644cea30" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37447 } }
I20260806 15:03:22.838167 8482 leader_election.cc:288] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30 [CANDIDATE]: Term 1 election: Requested vote from peers 8494a0194ec54e729ed1c8d22233b1e0 (127.0.236.67:40687), 4124281d1ce84becb16b06adfbd47976 (127.0.236.65:45143)
I20260806 15:03:22.838296 8388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ecb4b89b655486180007ab88ceae805" candidate_uuid: "23cab921e5cb4b44b028840f644cea30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8494a0194ec54e729ed1c8d22233b1e0"
I20260806 15:03:22.838325 8010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ecb4b89b655486180007ab88ceae805" candidate_uuid: "23cab921e5cb4b44b028840f644cea30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4124281d1ce84becb16b06adfbd47976"
I20260806 15:03:22.838364 8388 raft_consensus.cc:3037] T 1ecb4b89b655486180007ab88ceae805 P 8494a0194ec54e729ed1c8d22233b1e0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:22.838392 8010 raft_consensus.cc:3037] T 1ecb4b89b655486180007ab88ceae805 P 4124281d1ce84becb16b06adfbd47976 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:22.838994 8388 raft_consensus.cc:2454] T 1ecb4b89b655486180007ab88ceae805 P 8494a0194ec54e729ed1c8d22233b1e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cab921e5cb4b44b028840f644cea30 in term 1.
I20260806 15:03:22.839076 8010 raft_consensus.cc:2454] T 1ecb4b89b655486180007ab88ceae805 P 4124281d1ce84becb16b06adfbd47976 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cab921e5cb4b44b028840f644cea30 in term 1.
I20260806 15:03:22.839169 8110 leader_election.cc:302] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cab921e5cb4b44b028840f644cea30, 8494a0194ec54e729ed1c8d22233b1e0; no voters:
I20260806 15:03:22.839253 8482 raft_consensus.cc:2781] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:22.839387 8482 raft_consensus.cc:686] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30 [term 1 LEADER]: Becoming Leader. State: Replica: 23cab921e5cb4b44b028840f644cea30, State: Running, Role: LEADER
I20260806 15:03:22.839469 8482 consensus_queue.cc:237] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8494a0194ec54e729ed1c8d22233b1e0" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 40687 } } peers { permanent_uuid: "4124281d1ce84becb16b06adfbd47976" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45143 } } peers { permanent_uuid: "23cab921e5cb4b44b028840f644cea30" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37447 } }
I20260806 15:03:22.840190 7791 catalog_manager.cc:5495] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30 reported cstate change: term changed from 0 to 1, leader changed from <none> to 23cab921e5cb4b44b028840f644cea30 (127.0.236.66). New cstate: current_term: 1 leader_uuid: "23cab921e5cb4b44b028840f644cea30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8494a0194ec54e729ed1c8d22233b1e0" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 40687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4124281d1ce84becb16b06adfbd47976" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cab921e5cb4b44b028840f644cea30" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37447 } health_report { overall_health: HEALTHY } } }
W20260806 15:03:22.843724 8264 tablet.cc:2369] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:03:22.866712 8470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.865958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35924 (local address 127.0.236.66:37447)
0806 15:03:22.866008 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:22.866009 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:22.866017 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:22.866065 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:22.866066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:22.866075 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:22.866119 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:22.866581 (+ 462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:22.866582 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:22.866582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:03:22.866584 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:22.866593 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:03:22.866596 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:22.866628 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:22.866663 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 15:03:22.866671 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 15:03:22.868147 8388 raft_consensus.cc:1261] T 1ecb4b89b655486180007ab88ceae805 P 8494a0194ec54e729ed1c8d22233b1e0 [term 1 FOLLOWER]: Refusing update from remote peer 23cab921e5cb4b44b028840f644cea30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:22.868160 8010 raft_consensus.cc:1261] T 1ecb4b89b655486180007ab88ceae805 P 4124281d1ce84becb16b06adfbd47976 [term 1 FOLLOWER]: Refusing update from remote peer 23cab921e5cb4b44b028840f644cea30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:22.868446 8482 consensus_queue.cc:1035] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4124281d1ce84becb16b06adfbd47976" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:22.868580 8487 consensus_queue.cc:1035] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8494a0194ec54e729ed1c8d22233b1e0" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 40687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:22.870460 8493 mvcc.cc:205] Tried to move back new op lower bound from 7315573157345816576 to 7315573157231034368. Current Snapshot: MvccSnapshot[applied={T|T < 7315573157345816576}]
I20260806 15:03:22.870823 8494 mvcc.cc:205] Tried to move back new op lower bound from 7315573157345816576 to 7315573157231034368. Current Snapshot: MvccSnapshot[applied={T|T < 7315573157345816576}]
I20260806 15:03:22.871335 8495 mvcc.cc:205] Tried to move back new op lower bound from 7315573157345816576 to 7315573157231034368. Current Snapshot: MvccSnapshot[applied={T|T < 7315573157345816576}]
I20260806 15:03:22.872696 7787 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:22.872778 7787 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:22.873389 7787 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cde00388a34a405abcf7262e0d8d0fed: Sending DeleteTablet for 3 replicas of tablet 1ecb4b89b655486180007ab88ceae805
I20260806 15:03:22.873709 7977 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ecb4b89b655486180007ab88ceae805 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:22 UTC) from {username='slave'} at 127.0.0.1:46082
I20260806 15:03:22.873730 8361 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ecb4b89b655486180007ab88ceae805 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:22 UTC) from {username='slave'} at 127.0.0.1:47534
I20260806 15:03:22.873941 8506 tablet_replica.cc:330] T 1ecb4b89b655486180007ab88ceae805 P 4124281d1ce84becb16b06adfbd47976: stopping tablet replica
I20260806 15:03:22.874042 8506 raft_consensus.cc:2229] T 1ecb4b89b655486180007ab88ceae805 P 4124281d1ce84becb16b06adfbd47976 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:22.874123 8509 tablet_replica.cc:330] T 1ecb4b89b655486180007ab88ceae805 P 8494a0194ec54e729ed1c8d22233b1e0: stopping tablet replica
I20260806 15:03:22.874104 8170 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ecb4b89b655486180007ab88ceae805 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:22 UTC) from {username='slave'} at 127.0.0.1:35908
I20260806 15:03:22.874141 8506 raft_consensus.cc:2258] T 1ecb4b89b655486180007ab88ceae805 P 4124281d1ce84becb16b06adfbd47976 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:22.874200 8509 raft_consensus.cc:2229] T 1ecb4b89b655486180007ab88ceae805 P 8494a0194ec54e729ed1c8d22233b1e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:22.874228 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 7877
I20260806 15:03:22.874276 8509 raft_consensus.cc:2258] T 1ecb4b89b655486180007ab88ceae805 P 8494a0194ec54e729ed1c8d22233b1e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:22.878187 8509 ts_tablet_manager.cc:1905] T 1ecb4b89b655486180007ab88ceae805 P 8494a0194ec54e729ed1c8d22233b1e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:22.879530 8509 ts_tablet_manager.cc:1918] T 1ecb4b89b655486180007ab88ceae805 P 8494a0194ec54e729ed1c8d22233b1e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:03:22.879595 8509 log.cc:1196] T 1ecb4b89b655486180007ab88ceae805 P 8494a0194ec54e729ed1c8d22233b1e0: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/1ecb4b89b655486180007ab88ceae805
I20260806 15:03:22.879807 8509 ts_tablet_manager.cc:1939] T 1ecb4b89b655486180007ab88ceae805 P 8494a0194ec54e729ed1c8d22233b1e0: Deleting consensus metadata
I20260806 15:03:22.880121 7767 catalog_manager.cc:4841] TS 8494a0194ec54e729ed1c8d22233b1e0 (127.0.236.67:40687): tablet 1ecb4b89b655486180007ab88ceae805 (table test-table [id=3b84de903d1c4e99ab9a991cea209dca]) successfully deleted
I20260806 15:03:22.885754 8510 tablet_replica.cc:330] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30: stopping tablet replica
I20260806 15:03:22.885936 8510 raft_consensus.cc:2229] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30 [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:03:22.886071 8510 raft_consensus.cc:2258] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:22.886444 8510 ts_tablet_manager.cc:1905] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:22.888069 8510 ts_tablet_manager.cc:1918] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:03:22.888141 8510 log.cc:1196] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028580791223-945-0/minicluster-data/ts-1/wal/wals/1ecb4b89b655486180007ab88ceae805
I20260806 15:03:22.888463 8510 ts_tablet_manager.cc:1939] T 1ecb4b89b655486180007ab88ceae805 P 23cab921e5cb4b44b028840f644cea30: Deleting consensus metadata
I20260806 15:03:22.889058 7765 catalog_manager.cc:4841] TS 23cab921e5cb4b44b028840f644cea30 (127.0.236.66:37447): tablet 1ecb4b89b655486180007ab88ceae805 (table test-table [id=3b84de903d1c4e99ab9a991cea209dca]) successfully deleted
W20260806 15:03:22.889659 7766 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.236.65:45143 (error 108)
I20260806 15:03:22.889920 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 8086
I20260806 15:03:22.892715 8464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:22.890102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:45143 (local address 127.0.0.1:46086)
0806 15:03:22.892615 (+ 2513us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:22.892678 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.65:45143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2488}
W20260806 15:03:22.898348 7766 catalog_manager.cc:4584] TS 4124281d1ce84becb16b06adfbd47976 (127.0.236.65:45143): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1ecb4b89b655486180007ab88ceae805: Network error: Client connection negotiation failed: client connection to 127.0.236.65:45143: connect: Connection refused (error 111)
I20260806 15:03:22.903110 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 8266
I20260806 15:03:22.910104 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 7743
2026-08-06T15:03:22Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (710 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 15:03:22.931269 945 mini_oidc.cc:152] Starting JWKS server
I20260806 15:03:22.932436 945 webserver.cc:460] Webserver started at https://127.0.0.1:43867/ using document root <none> and password file <none>
I20260806 15:03:22.932511 945 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 15:03:22.932638 945 webserver.cc:460] Webserver started at http://127.0.0.1:37543/ using document root <none> and password file <none>
2026-08-06T15:03:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:03:22Z Disabled control of system clock
I20260806 15:03:22.940434 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:39583
--webserver_interface=127.0.236.126
--webserver_port=0
--builtin_ntp_servers=127.0.236.84:41517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:39583
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43867/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028580791223-945-0/testchainca.cert with env {}
W20260806 15:03:23.017755 8525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:23.017956 8525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:23.017976 8525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:23.019109 8525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:03:23.019148 8525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:23.019160 8525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:23.019182 8525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:03:23.019197 8525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 15:03:23.019219 8525 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 15:03:23.019239 8525 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43867/jwks.json
I20260806 15:03:23.020804 8525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:41517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:39583
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:39583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43867/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028580791223-945-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:23.021024 8525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:23.021224 8525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:23.025802 8535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:23.025825 8537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:23.025815 8532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:23.025866 8525 server_base.cc:1034] running on GCE node
I20260806 15:03:23.026290 8525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:23.026541 8525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:23.027686 8525 hybrid_clock.cc:648] HybridClock initialized: now 1786028603027675 us; error 30 us; skew 500 ppm
I20260806 15:03:23.028817 8525 webserver.cc:460] Webserver started at http://127.0.236.126:41983/ using document root <none> and password file <none>
I20260806 15:03:23.029031 8525 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:23.029074 8525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:23.029192 8525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:23.030179 8525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "d80d942905d945d4a94b6fdb236df495"
format_stamp: "Formatted at 2026-08-06 15:03:23 on dist-test-slave-tc2s"
server_key: "009c71c6f2abd9cfcc27a20917e6c76e"
server_key_iv: "a0b38af2868db5c68b48bb7abf888896"
server_key_version: "encryptionkey@0"
I20260806 15:03:23.030555 8525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "d80d942905d945d4a94b6fdb236df495"
format_stamp: "Formatted at 2026-08-06 15:03:23 on dist-test-slave-tc2s"
server_key: "009c71c6f2abd9cfcc27a20917e6c76e"
server_key_iv: "a0b38af2868db5c68b48bb7abf888896"
server_key_version: "encryptionkey@0"
I20260806 15:03:23.031725 8525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:03:23.032378 8544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:23.032505 8525 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:23.032578 8525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "d80d942905d945d4a94b6fdb236df495"
format_stamp: "Formatted at 2026-08-06 15:03:23 on dist-test-slave-tc2s"
server_key: "009c71c6f2abd9cfcc27a20917e6c76e"
server_key_iv: "a0b38af2868db5c68b48bb7abf888896"
server_key_version: "encryptionkey@0"
I20260806 15:03:23.032641 8525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:23.044170 8525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:23.044425 8525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:23.044536 8525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:23.048166 8525 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:39583
I20260806 15:03:23.048194 8612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:39583 every 8 connection(s)
I20260806 15:03:23.048470 8525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:03:23.049434 8613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:23.051570 8613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d80d942905d945d4a94b6fdb236df495: Bootstrap starting.
I20260806 15:03:23.052095 8613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d80d942905d945d4a94b6fdb236df495: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:23.052350 8613 log.cc:824] T 00000000000000000000000000000000 P d80d942905d945d4a94b6fdb236df495: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:23.053014 8613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d80d942905d945d4a94b6fdb236df495: No bootstrap required, opened a new log
I20260806 15:03:23.054042 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 8525
I20260806 15:03:23.054142 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:03:23.054296 945 external_mini_cluster.cc:1428] Setting key 2ab65becd881f3e5e60d88233dcced44
I20260806 15:03:23.054813 8613 raft_consensus.cc:348] T 00000000000000000000000000000000 P d80d942905d945d4a94b6fdb236df495 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d80d942905d945d4a94b6fdb236df495" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39583 } }
I20260806 15:03:23.054979 8613 raft_consensus.cc:374] T 00000000000000000000000000000000 P d80d942905d945d4a94b6fdb236df495 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:23.055022 8613 raft_consensus.cc:729] T 00000000000000000000000000000000 P d80d942905d945d4a94b6fdb236df495 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d80d942905d945d4a94b6fdb236df495, State: Initialized, Role: FOLLOWER
I20260806 15:03:23.055137 8613 consensus_queue.cc:260] T 00000000000000000000000000000000 P d80d942905d945d4a94b6fdb236df495 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d80d942905d945d4a94b6fdb236df495" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39583 } }
I20260806 15:03:23.055207 8613 raft_consensus.cc:388] T 00000000000000000000000000000000 P d80d942905d945d4a94b6fdb236df495 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:03:23.055243 8613 raft_consensus.cc:482] T 00000000000000000000000000000000 P d80d942905d945d4a94b6fdb236df495 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:03:23.055289 8613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d80d942905d945d4a94b6fdb236df495 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:23.055866 8613 raft_consensus.cc:504] T 00000000000000000000000000000000 P d80d942905d945d4a94b6fdb236df495 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d80d942905d945d4a94b6fdb236df495" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39583 } }
I20260806 15:03:23.055956 8613 leader_election.cc:302] T 00000000000000000000000000000000 P d80d942905d945d4a94b6fdb236df495 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d80d942905d945d4a94b6fdb236df495; no voters:
I20260806 15:03:23.056097 8613 leader_election.cc:288] T 00000000000000000000000000000000 P d80d942905d945d4a94b6fdb236df495 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:03:23.056155 8625 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d80d942905d945d4a94b6fdb236df495 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:23.056267 8625 raft_consensus.cc:686] T 00000000000000000000000000000000 P d80d942905d945d4a94b6fdb236df495 [term 1 LEADER]: Becoming Leader. State: Replica: d80d942905d945d4a94b6fdb236df495, State: Running, Role: LEADER
I20260806 15:03:23.056354 8613 sys_catalog.cc:564] T 00000000000000000000000000000000 P d80d942905d945d4a94b6fdb236df495 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:03:23.056346 8625 consensus_queue.cc:237] T 00000000000000000000000000000000 P d80d942905d945d4a94b6fdb236df495 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d80d942905d945d4a94b6fdb236df495" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39583 } }
I20260806 15:03:23.056757 8627 sys_catalog.cc:455] T 00000000000000000000000000000000 P d80d942905d945d4a94b6fdb236df495 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d80d942905d945d4a94b6fdb236df495. Latest consensus state: current_term: 1 leader_uuid: "d80d942905d945d4a94b6fdb236df495" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d80d942905d945d4a94b6fdb236df495" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39583 } } }
I20260806 15:03:23.056845 8627 sys_catalog.cc:458] T 00000000000000000000000000000000 P d80d942905d945d4a94b6fdb236df495 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:23.057003 8628 sys_catalog.cc:455] T 00000000000000000000000000000000 P d80d942905d945d4a94b6fdb236df495 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d80d942905d945d4a94b6fdb236df495" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d80d942905d945d4a94b6fdb236df495" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39583 } } }
I20260806 15:03:23.057055 8628 sys_catalog.cc:458] T 00000000000000000000000000000000 P d80d942905d945d4a94b6fdb236df495 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:23.057235 8633 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:03:23.057686 8633 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:03:23.060144 8624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:23.054732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54530 (local address 127.0.236.126:39583)
0806 15:03:23.054938 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:23.054946 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:23.054979 (+ 33us) server_negotiation.cc:407] Connection header received
0806 15:03:23.055092 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:23.055099 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:23.055165 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:23.055302 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:23.055627 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:23.057433 (+ 1806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:23.058487 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:23.058855 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:23.058933 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:23.059533 (+ 600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:23.059537 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:23.059537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:03:23.059692 (+ 155us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:23.059725 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:03:23.059735 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:23.059781 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:23.059878 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 15:03:23.059937 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":81,"threads_started":1}
I20260806 15:03:23.060993 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:03:23.066752 8633 catalog_manager.cc:1347] Generated new cluster ID: ea9a3a1bbac1498eb4856286a9fa4a2c
I20260806 15:03:23.066805 8633 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:03:23.074445 8633 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:03:23.074898 8633 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:03:23.083882 8633 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d80d942905d945d4a94b6fdb236df495: Generated new TSK 0
I20260806 15:03:23.084033 8633 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 15:03:23.106200 8660 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 15:03:23.106320 8660 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 15:03:23.106503 8624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:23.089506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54546 (local address 127.0.236.126:39583)
0806 15:03:23.089544 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:23.089546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:23.089626 (+ 80us) server_negotiation.cc:407] Connection header received
0806 15:03:23.089724 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:23.089725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:23.089734 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:23.089786 (+ 52us) server_negotiation.cc:226] Negotiated authn=JWT
0806 15:03:23.089971 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:23.090442 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:23.091534 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:23.091715 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:23.091755 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:23.091958 (+ 203us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 15:03:23.106300 (+ 14342us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43867/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 15:03:23.106443 (+ 143us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54546: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43867/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 15:03:23.106545 8624 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54546: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43867/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 15:03:23.106995 8658 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:23.089395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39583 (local address 127.0.0.1:54546)
0806 15:03:23.089558 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:23.089570 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:23.089639 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:23.089809 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:23.089810 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:23.089812 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 15:03:23.089883 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:23.089885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:23.090515 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:23.090516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:23.091464 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:23.091468 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:23.091777 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:23.091778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:23.091850 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:23.091873 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 15:03:23.106474 (+ 14601us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43867/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 15:03:23.106592 (+ 118us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.236.126:39583: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43867/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.0.236.126:39583: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43867/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 15:03:23.107364 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 8525
2026-08-06T15:03:23Z chronyd exiting
I20260806 15:03:23.151997 945 test_util.cc:175] -----------------------------------------------
I20260806 15:03:23.152063 945 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028580791223-945-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (221 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 15:03:23.152817 945 mini_oidc.cc:152] Starting JWKS server
I20260806 15:03:23.153455 945 webserver.cc:460] Webserver started at https://127.0.0.1:35573/ using document root <none> and password file <none>
I20260806 15:03:23.153496 945 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 15:03:23.153631 945 webserver.cc:460] Webserver started at http://127.0.0.1:34869/ using document root <none> and password file <none>
2026-08-06T15:03:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:03:23Z Disabled control of system clock
I20260806 15:03:23.161316 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:40333
--webserver_interface=127.0.236.126
--webserver_port=0
--builtin_ntp_servers=127.0.236.84:46851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:40333
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35573/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028580791223-945-0/testchainca.cert with env {}
W20260806 15:03:23.238960 8669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:23.239132 8669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:23.239152 8669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:23.240223 8669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:03:23.240257 8669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:23.240267 8669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:23.240278 8669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:03:23.240291 8669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 15:03:23.240302 8669 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 15:03:23.240317 8669 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35573/jwks.json
I20260806 15:03:23.241694 8669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:46851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:40333
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:40333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35573/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028580791223-945-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:23.241865 8669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:23.242026 8669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:23.246353 8677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:23.246376 8678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:23.246378 8669 server_base.cc:1034] running on GCE node
W20260806 15:03:23.246361 8680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:23.246744 8669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:23.247001 8669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:23.248142 8669 hybrid_clock.cc:648] HybridClock initialized: now 1786028603248127 us; error 27 us; skew 500 ppm
I20260806 15:03:23.256283 8669 webserver.cc:460] Webserver started at http://127.0.236.126:40173/ using document root <none> and password file <none>
I20260806 15:03:23.256475 8669 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:23.256521 8669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:23.256634 8669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:23.257565 8669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "226226dcd94b49a5aa25e8cfeb27ca46"
format_stamp: "Formatted at 2026-08-06 15:03:23 on dist-test-slave-tc2s"
server_key: "5ca0fe5806a7728832776b412b94b55e"
server_key_iv: "b701fa732ba732fa985f204dd9a03e41"
server_key_version: "encryptionkey@0"
I20260806 15:03:23.257864 8669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "226226dcd94b49a5aa25e8cfeb27ca46"
format_stamp: "Formatted at 2026-08-06 15:03:23 on dist-test-slave-tc2s"
server_key: "5ca0fe5806a7728832776b412b94b55e"
server_key_iv: "b701fa732ba732fa985f204dd9a03e41"
server_key_version: "encryptionkey@0"
I20260806 15:03:23.259004 8669 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:23.259619 8690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:23.259763 8669 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:23.259841 8669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "226226dcd94b49a5aa25e8cfeb27ca46"
format_stamp: "Formatted at 2026-08-06 15:03:23 on dist-test-slave-tc2s"
server_key: "5ca0fe5806a7728832776b412b94b55e"
server_key_iv: "b701fa732ba732fa985f204dd9a03e41"
server_key_version: "encryptionkey@0"
I20260806 15:03:23.259896 8669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:23.275079 8669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:23.275305 8669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:23.275424 8669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:23.279125 8669 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:40333
I20260806 15:03:23.279156 8762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:40333 every 8 connection(s)
I20260806 15:03:23.279426 8669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:03:23.280297 8763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:23.282397 8763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 226226dcd94b49a5aa25e8cfeb27ca46: Bootstrap starting.
I20260806 15:03:23.282904 8763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 226226dcd94b49a5aa25e8cfeb27ca46: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:23.283129 8763 log.cc:824] T 00000000000000000000000000000000 P 226226dcd94b49a5aa25e8cfeb27ca46: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:23.283684 8763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 226226dcd94b49a5aa25e8cfeb27ca46: No bootstrap required, opened a new log
I20260806 15:03:23.284948 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 8669
I20260806 15:03:23.285039 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:03:23.285190 945 external_mini_cluster.cc:1428] Setting key 768ad4722c8d58a2185d416b01be9f74
I20260806 15:03:23.285241 8763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 226226dcd94b49a5aa25e8cfeb27ca46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "226226dcd94b49a5aa25e8cfeb27ca46" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 40333 } }
I20260806 15:03:23.285395 8763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 226226dcd94b49a5aa25e8cfeb27ca46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:23.285431 8763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 226226dcd94b49a5aa25e8cfeb27ca46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 226226dcd94b49a5aa25e8cfeb27ca46, State: Initialized, Role: FOLLOWER
I20260806 15:03:23.285554 8763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 226226dcd94b49a5aa25e8cfeb27ca46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "226226dcd94b49a5aa25e8cfeb27ca46" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 40333 } }
I20260806 15:03:23.285635 8763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 226226dcd94b49a5aa25e8cfeb27ca46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:03:23.285673 8763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 226226dcd94b49a5aa25e8cfeb27ca46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:03:23.285719 8763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 226226dcd94b49a5aa25e8cfeb27ca46 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:23.286338 8763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 226226dcd94b49a5aa25e8cfeb27ca46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "226226dcd94b49a5aa25e8cfeb27ca46" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 40333 } }
I20260806 15:03:23.286468 8763 leader_election.cc:302] T 00000000000000000000000000000000 P 226226dcd94b49a5aa25e8cfeb27ca46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 226226dcd94b49a5aa25e8cfeb27ca46; no voters:
I20260806 15:03:23.286612 8763 leader_election.cc:288] T 00000000000000000000000000000000 P 226226dcd94b49a5aa25e8cfeb27ca46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:03:23.286650 8771 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 226226dcd94b49a5aa25e8cfeb27ca46 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:23.286789 8771 raft_consensus.cc:686] T 00000000000000000000000000000000 P 226226dcd94b49a5aa25e8cfeb27ca46 [term 1 LEADER]: Becoming Leader. State: Replica: 226226dcd94b49a5aa25e8cfeb27ca46, State: Running, Role: LEADER
I20260806 15:03:23.286856 8763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 226226dcd94b49a5aa25e8cfeb27ca46 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:03:23.286883 8771 consensus_queue.cc:237] T 00000000000000000000000000000000 P 226226dcd94b49a5aa25e8cfeb27ca46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "226226dcd94b49a5aa25e8cfeb27ca46" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 40333 } }
I20260806 15:03:23.287206 8771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 226226dcd94b49a5aa25e8cfeb27ca46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 226226dcd94b49a5aa25e8cfeb27ca46. Latest consensus state: current_term: 1 leader_uuid: "226226dcd94b49a5aa25e8cfeb27ca46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "226226dcd94b49a5aa25e8cfeb27ca46" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 40333 } } }
I20260806 15:03:23.287278 8771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 226226dcd94b49a5aa25e8cfeb27ca46 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:23.287425 8772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 226226dcd94b49a5aa25e8cfeb27ca46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "226226dcd94b49a5aa25e8cfeb27ca46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "226226dcd94b49a5aa25e8cfeb27ca46" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 40333 } } }
I20260806 15:03:23.287487 8772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 226226dcd94b49a5aa25e8cfeb27ca46 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:23.287983 8781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:03:23.288419 8781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:03:23.289958 8781 catalog_manager.cc:1347] Generated new cluster ID: 7661e76b4d6148b5a3c2a5d2efd743b8
I20260806 15:03:23.290009 8781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:03:23.294785 8769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:23.285605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33786 (local address 127.0.236.126:40333)
0806 15:03:23.285869 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:23.285877 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:23.285930 (+ 53us) server_negotiation.cc:407] Connection header received
0806 15:03:23.286042 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:23.286049 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:23.286124 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:23.286263 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:23.286429 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:23.288118 (+ 1689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:23.288960 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:23.289529 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:23.289671 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:23.294127 (+ 4456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:23.294132 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:23.294132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:03:23.294326 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:23.294367 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:03:23.294382 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:23.294454 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:23.294510 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 15:03:23.294595 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":90,"threads_started":1}
I20260806 15:03:23.295771 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:03:23.310076 8781 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:03:23.310549 8781 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:03:23.314560 8781 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 226226dcd94b49a5aa25e8cfeb27ca46: Generated new TSK 0
I20260806 15:03:23.314695 8781 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 15:03:23.332736 8807 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 15:03:23.332788 8807 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 15:03:23.333096 8769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:23.319335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33800 (local address 127.0.236.126:40333)
0806 15:03:23.319377 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:23.319379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:23.319464 (+ 85us) server_negotiation.cc:407] Connection header received
0806 15:03:23.319510 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:23.319511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:23.319522 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:23.319600 (+ 78us) server_negotiation.cc:226] Negotiated authn=JWT
0806 15:03:23.319700 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:23.320450 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:23.321171 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:23.321468 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:23.321533 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:23.321661 (+ 128us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 15:03:23.332909 (+ 11248us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35573/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 15:03:23.333033 (+ 124us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33800: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35573/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 15:03:23.333148 8769 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33800: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35573/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 15:03:23.333175 8806 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:23.319264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:40333 (local address 127.0.0.1:33800)
0806 15:03:23.319408 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:23.319418 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:23.319465 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:23.319601 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:23.319602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:23.319603 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0806 15:03:23.319648 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:23.319650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:23.320514 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:23.320514 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:23.321099 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:23.321101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:23.321538 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:23.321539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:23.321582 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:23.321586 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 15:03:23.333011 (+ 11425us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35573/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 15:03:23.333102 (+ 91us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.236.126:40333: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35573/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
I20260806 15:03:23.333410 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 8669
2026-08-06T15:03:23Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (223 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 15:03:23.375788 945 mini_oidc.cc:152] Starting JWKS server
I20260806 15:03:23.376400 945 webserver.cc:460] Webserver started at https://127.0.0.1:36041/ using document root <none> and password file <none>
I20260806 15:03:23.376441 945 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 15:03:23.376582 945 webserver.cc:460] Webserver started at http://127.0.0.1:34317/ using document root <none> and password file <none>
2026-08-06T15:03:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:03:23Z Disabled control of system clock
I20260806 15:03:23.384093 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:44055
--webserver_interface=127.0.236.126
--webserver_port=0
--builtin_ntp_servers=127.0.236.84:37097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:44055
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36041/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 15:03:23.465291 8815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:23.465484 8815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:23.465514 8815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:23.466652 8815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:03:23.466691 8815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:23.466708 8815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:23.466724 8815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:03:23.466740 8815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 15:03:23.466770 8815 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 15:03:23.466786 8815 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36041/jwks.json
I20260806 15:03:23.468150 8815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:37097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:44055
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:44055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36041/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:23.468384 8815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:23.468619 8815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:23.472982 8823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:23.473008 8827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:23.473171 8815 server_base.cc:1034] running on GCE node
W20260806 15:03:23.472990 8824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:23.473500 8815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:23.473762 8815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:23.474896 8815 hybrid_clock.cc:648] HybridClock initialized: now 1786028603474878 us; error 32 us; skew 500 ppm
I20260806 15:03:23.476003 8815 webserver.cc:460] Webserver started at http://127.0.236.126:45127/ using document root <none> and password file <none>
I20260806 15:03:23.476214 8815 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:23.476256 8815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:23.476380 8815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:23.477347 8815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "f85158a6826e4ef1baa8b52a797e5871"
format_stamp: "Formatted at 2026-08-06 15:03:23 on dist-test-slave-tc2s"
server_key: "0872a15443c21cfc8054c6fdbc622420"
server_key_iv: "718e86dd95ea7dc41418505fb222c310"
server_key_version: "encryptionkey@0"
I20260806 15:03:23.477677 8815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "f85158a6826e4ef1baa8b52a797e5871"
format_stamp: "Formatted at 2026-08-06 15:03:23 on dist-test-slave-tc2s"
server_key: "0872a15443c21cfc8054c6fdbc622420"
server_key_iv: "718e86dd95ea7dc41418505fb222c310"
server_key_version: "encryptionkey@0"
I20260806 15:03:23.478838 8815 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:03:23.479467 8838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:23.479601 8815 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:03:23.479686 8815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "f85158a6826e4ef1baa8b52a797e5871"
format_stamp: "Formatted at 2026-08-06 15:03:23 on dist-test-slave-tc2s"
server_key: "0872a15443c21cfc8054c6fdbc622420"
server_key_iv: "718e86dd95ea7dc41418505fb222c310"
server_key_version: "encryptionkey@0"
I20260806 15:03:23.479753 8815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:23.499819 8815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:23.500109 8815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:23.500233 8815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:23.504094 8815 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:44055
I20260806 15:03:23.504127 8918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:44055 every 8 connection(s)
I20260806 15:03:23.504431 8815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:03:23.505064 8919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:23.507973 8919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f85158a6826e4ef1baa8b52a797e5871: Bootstrap starting.
I20260806 15:03:23.508023 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 8815
I20260806 15:03:23.508112 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:03:23.508271 945 external_mini_cluster.cc:1428] Setting key 22588b7e69e836d6aa7eecd796480e0a
I20260806 15:03:23.508641 8919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f85158a6826e4ef1baa8b52a797e5871: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:23.508975 8919 log.cc:824] T 00000000000000000000000000000000 P f85158a6826e4ef1baa8b52a797e5871: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:23.509682 8919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f85158a6826e4ef1baa8b52a797e5871: No bootstrap required, opened a new log
I20260806 15:03:23.511552 8919 raft_consensus.cc:348] T 00000000000000000000000000000000 P f85158a6826e4ef1baa8b52a797e5871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f85158a6826e4ef1baa8b52a797e5871" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44055 } }
I20260806 15:03:23.511713 8919 raft_consensus.cc:374] T 00000000000000000000000000000000 P f85158a6826e4ef1baa8b52a797e5871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:23.511742 8919 raft_consensus.cc:729] T 00000000000000000000000000000000 P f85158a6826e4ef1baa8b52a797e5871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f85158a6826e4ef1baa8b52a797e5871, State: Initialized, Role: FOLLOWER
I20260806 15:03:23.511863 8919 consensus_queue.cc:260] T 00000000000000000000000000000000 P f85158a6826e4ef1baa8b52a797e5871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f85158a6826e4ef1baa8b52a797e5871" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44055 } }
I20260806 15:03:23.511940 8919 raft_consensus.cc:388] T 00000000000000000000000000000000 P f85158a6826e4ef1baa8b52a797e5871 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:03:23.511961 8919 raft_consensus.cc:482] T 00000000000000000000000000000000 P f85158a6826e4ef1baa8b52a797e5871 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:03:23.512003 8919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f85158a6826e4ef1baa8b52a797e5871 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:23.512530 8919 raft_consensus.cc:504] T 00000000000000000000000000000000 P f85158a6826e4ef1baa8b52a797e5871 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f85158a6826e4ef1baa8b52a797e5871" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44055 } }
I20260806 15:03:23.512639 8919 leader_election.cc:302] T 00000000000000000000000000000000 P f85158a6826e4ef1baa8b52a797e5871 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f85158a6826e4ef1baa8b52a797e5871; no voters:
I20260806 15:03:23.512785 8919 leader_election.cc:288] T 00000000000000000000000000000000 P f85158a6826e4ef1baa8b52a797e5871 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:03:23.512950 8926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f85158a6826e4ef1baa8b52a797e5871 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:23.513043 8919 sys_catalog.cc:564] T 00000000000000000000000000000000 P f85158a6826e4ef1baa8b52a797e5871 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:03:23.513137 8926 raft_consensus.cc:686] T 00000000000000000000000000000000 P f85158a6826e4ef1baa8b52a797e5871 [term 1 LEADER]: Becoming Leader. State: Replica: f85158a6826e4ef1baa8b52a797e5871, State: Running, Role: LEADER
I20260806 15:03:23.513224 8926 consensus_queue.cc:237] T 00000000000000000000000000000000 P f85158a6826e4ef1baa8b52a797e5871 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f85158a6826e4ef1baa8b52a797e5871" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44055 } }
I20260806 15:03:23.513664 8928 sys_catalog.cc:455] T 00000000000000000000000000000000 P f85158a6826e4ef1baa8b52a797e5871 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f85158a6826e4ef1baa8b52a797e5871. Latest consensus state: current_term: 1 leader_uuid: "f85158a6826e4ef1baa8b52a797e5871" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f85158a6826e4ef1baa8b52a797e5871" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44055 } } }
I20260806 15:03:23.513748 8928 sys_catalog.cc:458] T 00000000000000000000000000000000 P f85158a6826e4ef1baa8b52a797e5871 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:23.513984 8936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:03:23.514005 8925 sys_catalog.cc:455] T 00000000000000000000000000000000 P f85158a6826e4ef1baa8b52a797e5871 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f85158a6826e4ef1baa8b52a797e5871" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f85158a6826e4ef1baa8b52a797e5871" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44055 } } }
I20260806 15:03:23.514034 8922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:23.508739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50106 (local address 127.0.236.126:44055)
0806 15:03:23.508924 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:23.508931 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:23.508957 (+ 26us) server_negotiation.cc:407] Connection header received
0806 15:03:23.509449 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:23.509458 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:23.509517 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:23.509629 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:23.509866 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:23.510777 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:23.511848 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:23.512351 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:23.512446 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:23.513197 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:23.513201 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:23.513202 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:03:23.513388 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:23.513430 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:03:23.513485 (+ 55us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:23.513554 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:23.513734 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 15:03:23.513820 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
I20260806 15:03:23.514110 8925 sys_catalog.cc:458] T 00000000000000000000000000000000 P f85158a6826e4ef1baa8b52a797e5871 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:23.514930 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:03:23.515043 8936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:03:23.516436 8936 catalog_manager.cc:1347] Generated new cluster ID: d395986905304231ae3fe776cdb439bc
I20260806 15:03:23.516494 8936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:03:23.536847 8936 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:03:23.537511 8936 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:03:23.542905 8936 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f85158a6826e4ef1baa8b52a797e5871: Generated new TSK 0
I20260806 15:03:23.543071 8936 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:03:23.556519 8922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:23.553039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50118 (local address 127.0.236.126:44055)
0806 15:03:23.553080 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:23.553082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:23.553131 (+ 49us) server_negotiation.cc:407] Connection header received
0806 15:03:23.553186 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:23.553187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:23.553199 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:23.553269 (+ 70us) server_negotiation.cc:226] Negotiated authn=JWT
0806 15:03:23.553374 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:23.554171 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:23.554877 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:23.555202 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:23.555267 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:23.555431 (+ 164us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 15:03:23.556334 (+ 903us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36041/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 15:03:23.556455 (+ 121us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50118: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36041/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":25}
W20260806 15:03:23.556569 8922 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50118: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36041/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 15:03:23.558382 8959 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:23.552960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:44055 (local address 127.0.0.1:50118)
0806 15:03:23.553083 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:23.553092 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:23.553139 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:23.553274 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:23.553275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:23.553276 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0806 15:03:23.553323 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:23.553325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:23.554235 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:23.554235 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:23.554811 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:23.554812 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:23.555283 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:23.555284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:23.555322 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:23.555326 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 15:03:23.556427 (+ 1101us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36041/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 15:03:23.558285 (+ 1858us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.236.126:44055: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36041/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":1689,"thread_start_us":69,"threads_started":1}
I20260806 15:03:23.558988 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 8815
E20260806 15:03:23.576014 8960 webserver.cc:533] Webserver: sslize failed:
E20260806 15:03:23.576076 8960 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T15:03:23Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (228 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T15:03:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:03:23Z Disabled control of system clock
I20260806 15:03:23.612114 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:37441
--webserver_interface=127.0.236.126
--webserver_port=0
--builtin_ntp_servers=127.0.236.84:39049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:37441
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028580791223-945-0/testchainca.cert with env {}
W20260806 15:03:23.690984 8965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:23.691164 8965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:23.691195 8965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:23.692314 8965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:03:23.692354 8965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:23.692371 8965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:23.692387 8965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:03:23.692401 8965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 15:03:23.692418 8965 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 15:03:23.692440 8965 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 15:03:23.693909 8965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:39049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:37441
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:37441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028580791223-945-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:23.694147 8965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:23.694331 8965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:23.698709 8970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:23.698719 8971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:23.698781 8965 server_base.cc:1034] running on GCE node
W20260806 15:03:23.698719 8975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:23.699120 8965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:23.699368 8965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:23.700503 8965 hybrid_clock.cc:648] HybridClock initialized: now 1786028603700488 us; error 28 us; skew 500 ppm
I20260806 15:03:23.701568 8965 webserver.cc:460] Webserver started at http://127.0.236.126:46391/ using document root <none> and password file <none>
I20260806 15:03:23.701778 8965 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:23.701825 8965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:23.701972 8965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:23.702890 8965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "161ca3d11d834aa585c1fa04bd1727ea"
format_stamp: "Formatted at 2026-08-06 15:03:23 on dist-test-slave-tc2s"
server_key: "2a2d2208f40c2208ee7b8722fd3ae268"
server_key_iv: "82cee47565d033b593239ad8d1f68fc3"
server_key_version: "encryptionkey@0"
I20260806 15:03:23.703205 8965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "161ca3d11d834aa585c1fa04bd1727ea"
format_stamp: "Formatted at 2026-08-06 15:03:23 on dist-test-slave-tc2s"
server_key: "2a2d2208f40c2208ee7b8722fd3ae268"
server_key_iv: "82cee47565d033b593239ad8d1f68fc3"
server_key_version: "encryptionkey@0"
I20260806 15:03:23.704489 8965 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 15:03:23.705089 8984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:23.705237 8965 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:03:23.705313 8965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "161ca3d11d834aa585c1fa04bd1727ea"
format_stamp: "Formatted at 2026-08-06 15:03:23 on dist-test-slave-tc2s"
server_key: "2a2d2208f40c2208ee7b8722fd3ae268"
server_key_iv: "82cee47565d033b593239ad8d1f68fc3"
server_key_version: "encryptionkey@0"
I20260806 15:03:23.705374 8965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:23.739245 8965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:23.739506 8965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:23.739629 8965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:23.743312 8965 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:37441
I20260806 15:03:23.743348 9056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:37441 every 8 connection(s)
I20260806 15:03:23.743633 8965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:03:23.744516 9057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:23.746063 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 8965
I20260806 15:03:23.746157 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:03:23.746330 945 external_mini_cluster.cc:1428] Setting key 00070822de260822c451ad08d710c842
I20260806 15:03:23.746915 9057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 161ca3d11d834aa585c1fa04bd1727ea: Bootstrap starting.
I20260806 15:03:23.747495 9057 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 161ca3d11d834aa585c1fa04bd1727ea: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:23.747767 9057 log.cc:824] T 00000000000000000000000000000000 P 161ca3d11d834aa585c1fa04bd1727ea: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:23.748401 9057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 161ca3d11d834aa585c1fa04bd1727ea: No bootstrap required, opened a new log
I20260806 15:03:23.750164 9057 raft_consensus.cc:348] T 00000000000000000000000000000000 P 161ca3d11d834aa585c1fa04bd1727ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161ca3d11d834aa585c1fa04bd1727ea" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 37441 } }
I20260806 15:03:23.750293 9057 raft_consensus.cc:374] T 00000000000000000000000000000000 P 161ca3d11d834aa585c1fa04bd1727ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:23.750326 9057 raft_consensus.cc:729] T 00000000000000000000000000000000 P 161ca3d11d834aa585c1fa04bd1727ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 161ca3d11d834aa585c1fa04bd1727ea, State: Initialized, Role: FOLLOWER
I20260806 15:03:23.750454 9057 consensus_queue.cc:260] T 00000000000000000000000000000000 P 161ca3d11d834aa585c1fa04bd1727ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161ca3d11d834aa585c1fa04bd1727ea" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 37441 } }
I20260806 15:03:23.750519 9057 raft_consensus.cc:388] T 00000000000000000000000000000000 P 161ca3d11d834aa585c1fa04bd1727ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:03:23.750537 9057 raft_consensus.cc:482] T 00000000000000000000000000000000 P 161ca3d11d834aa585c1fa04bd1727ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:03:23.750572 9057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 161ca3d11d834aa585c1fa04bd1727ea [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:23.751096 9057 raft_consensus.cc:504] T 00000000000000000000000000000000 P 161ca3d11d834aa585c1fa04bd1727ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161ca3d11d834aa585c1fa04bd1727ea" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 37441 } }
I20260806 15:03:23.751156 9061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:23.746793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60092 (local address 127.0.236.126:37441)
0806 15:03:23.746963 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:23.746970 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:23.746993 (+ 23us) server_negotiation.cc:407] Connection header received
0806 15:03:23.747427 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:23.747433 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:23.747489 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:23.747613 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:23.747827 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:23.748730 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:23.749462 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:23.749853 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:23.749998 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:23.750599 (+ 601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:23.750603 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:23.750603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:03:23.750767 (+ 164us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:23.750799 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:03:23.750810 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:23.750863 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:23.750941 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 15:03:23.751019 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":61,"threads_started":1}
I20260806 15:03:23.751194 9057 leader_election.cc:302] T 00000000000000000000000000000000 P 161ca3d11d834aa585c1fa04bd1727ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 161ca3d11d834aa585c1fa04bd1727ea; no voters:
I20260806 15:03:23.751338 9057 leader_election.cc:288] T 00000000000000000000000000000000 P 161ca3d11d834aa585c1fa04bd1727ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:03:23.751382 9063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 161ca3d11d834aa585c1fa04bd1727ea [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:23.751524 9063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 161ca3d11d834aa585c1fa04bd1727ea [term 1 LEADER]: Becoming Leader. State: Replica: 161ca3d11d834aa585c1fa04bd1727ea, State: Running, Role: LEADER
I20260806 15:03:23.751605 9063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 161ca3d11d834aa585c1fa04bd1727ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161ca3d11d834aa585c1fa04bd1727ea" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 37441 } }
I20260806 15:03:23.751751 9057 sys_catalog.cc:564] T 00000000000000000000000000000000 P 161ca3d11d834aa585c1fa04bd1727ea [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:03:23.751919 9063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 161ca3d11d834aa585c1fa04bd1727ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 161ca3d11d834aa585c1fa04bd1727ea. Latest consensus state: current_term: 1 leader_uuid: "161ca3d11d834aa585c1fa04bd1727ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161ca3d11d834aa585c1fa04bd1727ea" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 37441 } } }
I20260806 15:03:23.751922 9065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 161ca3d11d834aa585c1fa04bd1727ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "161ca3d11d834aa585c1fa04bd1727ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161ca3d11d834aa585c1fa04bd1727ea" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 37441 } } }
I20260806 15:03:23.752003 9063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 161ca3d11d834aa585c1fa04bd1727ea [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:23.752018 9065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 161ca3d11d834aa585c1fa04bd1727ea [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:23.752223 9067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:03:23.752228 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:03:23.752707 9067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:03:23.754442 9067 catalog_manager.cc:1347] Generated new cluster ID: 356fc701bfcd48c29a91f45a34008d05
I20260806 15:03:23.754499 9067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:03:23.784065 9067 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:03:23.784727 9067 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:03:23.789947 9067 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 161ca3d11d834aa585c1fa04bd1727ea: Generated new TSK 0
I20260806 15:03:23.790107 9067 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:03:24.224596 9061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:23.796694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60108 (local address 127.0.236.126:37441)
0806 15:03:23.796745 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:23.796747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:23.796834 (+ 87us) server_negotiation.cc:407] Connection header received
0806 15:03:23.796893 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:23.796894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:23.796905 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:23.796980 (+ 75us) server_negotiation.cc:226] Negotiated authn=JWT
0806 15:03:23.797098 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:23.797586 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:23.798365 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:23.798578 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:23.798632 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:23.798787 (+ 155us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 15:03:24.224338 (+425551us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 15:03:24.224513 (+ 175us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60108: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":29}
W20260806 15:03:24.224684 9061 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60108: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 15:03:24.224709 9098 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:23.796633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:37441 (local address 127.0.0.1:60108)
0806 15:03:23.796773 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:23.796782 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:23.796836 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:23.796984 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:23.796985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:23.796987 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 15:03:23.797042 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:23.797044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:23.797661 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:23.797662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:23.798289 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:23.798292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:23.798649 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:23.798650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:23.798702 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:23.798706 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 15:03:24.224486 (+425780us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 15:03:24.224606 (+ 120us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.236.126:37441: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260806 15:03:24.225101 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 8965
2026-08-06T15:03:24Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (644 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (87 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (86 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:03:24 dist-test-slave-tc2s krb5kdc[9117](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:03:24 dist-test-slave-tc2s krb5kdc[9117](info): set up 2 sockets
Aug 06 15:03:24 dist-test-slave-tc2s krb5kdc[9117](info): commencing operation
krb5kdc: starting...
W20260806 15:03:25.511109 945 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:03:25 dist-test-slave-tc2s krb5kdc[9117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:03:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:03:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:03:25.618669 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:32801
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:35619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:32801
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:25.698194 9140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:25.698370 9140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:25.698390 9140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:25.699506 9140 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:03:25.699548 9140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:25.699560 9140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:25.699584 9140 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:03:25.699600 9140 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:03:25.701103 9140 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:35619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:32801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:32801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:25.701320 9140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:25.701512 9140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:25.705844 9146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:25.705878 9145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:25.705886 9140 server_base.cc:1034] running on GCE node
W20260806 15:03:25.705852 9148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:25.706283 9140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:25.706552 9140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:25.707696 9140 hybrid_clock.cc:648] HybridClock initialized: now 1786028605707683 us; error 26 us; skew 500 ppm
Aug 06 15:03:25 dist-test-slave-tc2s krb5kdc[9117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028605, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:25.710035 9140 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:03:25.710455 9140 webserver.cc:460] Webserver started at http://127.0.236.126:39933/ using document root <none> and password file <none>
I20260806 15:03:25.710670 9140 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:25.710716 9140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:25.710843 9140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:25.711794 9140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "9a4e0077d85f4680b4bc3b4a11e53aca"
format_stamp: "Formatted at 2026-08-06 15:03:25 on dist-test-slave-tc2s"
server_key: "740cefdb34dcceeee83eec28bdc50e8e"
server_key_iv: "1984d9eed5b6463ea87eba8f96877dc9"
server_key_version: "encryptionkey@0"
I20260806 15:03:25.712121 9140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "9a4e0077d85f4680b4bc3b4a11e53aca"
format_stamp: "Formatted at 2026-08-06 15:03:25 on dist-test-slave-tc2s"
server_key: "740cefdb34dcceeee83eec28bdc50e8e"
server_key_iv: "1984d9eed5b6463ea87eba8f96877dc9"
server_key_version: "encryptionkey@0"
I20260806 15:03:25.713282 9140 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:03:25.713917 9158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:25.714058 9140 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:25.714131 9140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "9a4e0077d85f4680b4bc3b4a11e53aca"
format_stamp: "Formatted at 2026-08-06 15:03:25 on dist-test-slave-tc2s"
server_key: "740cefdb34dcceeee83eec28bdc50e8e"
server_key_iv: "1984d9eed5b6463ea87eba8f96877dc9"
server_key_version: "encryptionkey@0"
I20260806 15:03:25.714197 9140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:25.740005 9140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:25.742025 9140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:25.742156 9140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:25.745786 9140 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:32801
I20260806 15:03:25.745841 9222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:32801 every 8 connection(s)
I20260806 15:03:25.746299 9140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:03:25.746919 9223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:25.749012 9223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a4e0077d85f4680b4bc3b4a11e53aca: Bootstrap starting.
I20260806 15:03:25.749548 9223 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a4e0077d85f4680b4bc3b4a11e53aca: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:25.749807 9223 log.cc:824] T 00000000000000000000000000000000 P 9a4e0077d85f4680b4bc3b4a11e53aca: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:25.750414 9223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a4e0077d85f4680b4bc3b4a11e53aca: No bootstrap required, opened a new log
I20260806 15:03:25.751894 9223 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a4e0077d85f4680b4bc3b4a11e53aca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a4e0077d85f4680b4bc3b4a11e53aca" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 32801 } }
I20260806 15:03:25.752031 9223 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a4e0077d85f4680b4bc3b4a11e53aca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:25.752064 9223 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a4e0077d85f4680b4bc3b4a11e53aca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a4e0077d85f4680b4bc3b4a11e53aca, State: Initialized, Role: FOLLOWER
I20260806 15:03:25.752194 9223 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a4e0077d85f4680b4bc3b4a11e53aca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a4e0077d85f4680b4bc3b4a11e53aca" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 32801 } }
I20260806 15:03:25.752264 9223 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a4e0077d85f4680b4bc3b4a11e53aca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:03:25.752285 9223 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a4e0077d85f4680b4bc3b4a11e53aca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:03:25.752329 9223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a4e0077d85f4680b4bc3b4a11e53aca [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:25.752650 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 9140
I20260806 15:03:25.752738 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:03:25.752899 945 external_mini_cluster.cc:1428] Setting key 5e26c5f11ef6e4c4c214c60297ef24a4
I20260806 15:03:25.753269 9223 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a4e0077d85f4680b4bc3b4a11e53aca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a4e0077d85f4680b4bc3b4a11e53aca" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 32801 } }
I20260806 15:03:25.753379 9223 leader_election.cc:302] T 00000000000000000000000000000000 P 9a4e0077d85f4680b4bc3b4a11e53aca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a4e0077d85f4680b4bc3b4a11e53aca; no voters:
I20260806 15:03:25.753535 9223 leader_election.cc:288] T 00000000000000000000000000000000 P 9a4e0077d85f4680b4bc3b4a11e53aca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:03:25.753572 9233 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a4e0077d85f4680b4bc3b4a11e53aca [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:25.753742 9233 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a4e0077d85f4680b4bc3b4a11e53aca [term 1 LEADER]: Becoming Leader. State: Replica: 9a4e0077d85f4680b4bc3b4a11e53aca, State: Running, Role: LEADER
I20260806 15:03:25.753775 9223 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a4e0077d85f4680b4bc3b4a11e53aca [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:03:25.753831 9233 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a4e0077d85f4680b4bc3b4a11e53aca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a4e0077d85f4680b4bc3b4a11e53aca" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 32801 } }
I20260806 15:03:25.754172 9233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a4e0077d85f4680b4bc3b4a11e53aca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a4e0077d85f4680b4bc3b4a11e53aca. Latest consensus state: current_term: 1 leader_uuid: "9a4e0077d85f4680b4bc3b4a11e53aca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a4e0077d85f4680b4bc3b4a11e53aca" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 32801 } } }
I20260806 15:03:25.754180 9234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a4e0077d85f4680b4bc3b4a11e53aca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a4e0077d85f4680b4bc3b4a11e53aca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a4e0077d85f4680b4bc3b4a11e53aca" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 32801 } } }
I20260806 15:03:25.754266 9233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a4e0077d85f4680b4bc3b4a11e53aca [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:25.754272 9234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a4e0077d85f4680b4bc3b4a11e53aca [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:25.754534 9242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:03:25.755033 9242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:03:25.756472 9242 catalog_manager.cc:1347] Generated new cluster ID: 1224ad993c424fed95da7a37cbd4ef90
I20260806 15:03:25.756526 9242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 15:03:25 dist-test-slave-tc2s krb5kdc[9117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:25.763569 9232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:25.753327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60082 (local address 127.0.236.126:32801)
0806 15:03:25.753497 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:25.753507 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:25.753695 (+ 188us) server_negotiation.cc:407] Connection header received
0806 15:03:25.753838 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:25.753845 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:25.753925 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:25.754058 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:25.755029 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:25.756179 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:25.757194 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:25.757586 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:25.757662 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:25.760237 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:25.760241 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:25.760241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:25.760252 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:25.762083 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:25.762455 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:25.762457 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:25.762458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:25.762563 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:25.762762 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:25.762762 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:25.762763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:25.763030 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:25.763182 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:25.763268 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 15:03:25.763386 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
I20260806 15:03:25.774137 9242 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:03:25.774807 9242 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:03:25.782058 9242 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a4e0077d85f4680b4bc3b4a11e53aca: Generated new TSK 0
I20260806 15:03:25.782210 9242 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.236.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
I20260806 15:03:25.812243 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:0
--local_ip_for_outbound_sockets=127.0.236.65
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:32801
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:35619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:25.890605 9265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:25.890799 9265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:25.890821 9265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:25.891930 9265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:25.891971 9265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:25.892000 9265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:03:25.893756 9265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:35619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:32801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:25.894042 9265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:25.894301 9265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:25.900013 9280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:25.900359 9274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:25.901978 9275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:25.902547 9265 server_base.cc:1034] running on GCE node
I20260806 15:03:25.902779 9265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:25.903054 9265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:25.904198 9265 hybrid_clock.cc:648] HybridClock initialized: now 1786028605904178 us; error 36 us; skew 500 ppm
Aug 06 15:03:25 dist-test-slave-tc2s krb5kdc[9117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028605, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:25.906632 9265 init.cc:377] Logged in from keytab as kudu/127.0.236.65@KRBTEST.COM (short username kudu)
I20260806 15:03:25.907035 9265 webserver.cc:460] Webserver started at http://127.0.236.65:34781/ using document root <none> and password file <none>
I20260806 15:03:25.907248 9265 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:25.907296 9265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:25.907430 9265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:25.908449 9265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-0/data/instance:
uuid: "659a747346c24036917799688c008ccc"
format_stamp: "Formatted at 2026-08-06 15:03:25 on dist-test-slave-tc2s"
server_key: "ed32cb320c128faed038911f8f2edc7a"
server_key_iv: "97710e560e50ebcdd199e2d1c3914eda"
server_key_version: "encryptionkey@0"
I20260806 15:03:25.908773 9265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-0/wal/instance:
uuid: "659a747346c24036917799688c008ccc"
format_stamp: "Formatted at 2026-08-06 15:03:25 on dist-test-slave-tc2s"
server_key: "ed32cb320c128faed038911f8f2edc7a"
server_key_iv: "97710e560e50ebcdd199e2d1c3914eda"
server_key_version: "encryptionkey@0"
I20260806 15:03:25.909998 9265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 15:03:25.910667 9288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:25.910825 9265 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:25.910904 9265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "659a747346c24036917799688c008ccc"
format_stamp: "Formatted at 2026-08-06 15:03:25 on dist-test-slave-tc2s"
server_key: "ed32cb320c128faed038911f8f2edc7a"
server_key_iv: "97710e560e50ebcdd199e2d1c3914eda"
server_key_version: "encryptionkey@0"
I20260806 15:03:25.910974 9265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:25.926127 9265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:25.928120 9265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:25.928236 9265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:25.928439 9265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:25.928700 9265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:25.928732 9265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:25.928752 9265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:25.928776 9265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:25.934222 9265 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:41667
I20260806 15:03:25.934301 9458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:41667 every 8 connection(s)
I20260806 15:03:25.934559 9265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:03:25.936425 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 9265
I20260806 15:03:25.936519 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-0/wal/instance
I20260806 15:03:25.936695 945 external_mini_cluster.cc:1428] Setting key c718e1182638a584fa12bb35a504f650
Aug 06 15:03:25 dist-test-slave-tc2s krb5kdc[9117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028605, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:25.943436 9232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:25.935199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:51329 (local address 127.0.236.126:32801)
0806 15:03:25.935247 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:25.935249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:25.935406 (+ 157us) server_negotiation.cc:407] Connection header received
0806 15:03:25.936324 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:25.936326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:25.936342 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:25.936395 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:25.937097 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:25.937655 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:25.938689 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:25.938899 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:25.938959 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:25.940901 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:25.940902 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:25.940903 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:25.940914 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:25.942193 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:25.942558 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:25.942559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:25.942560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:25.942615 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:25.942837 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:25.942838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:25.942838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:25.942959 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:25.943013 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:25.943264 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 15:03:25.943348 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 15:03:25.943593 9462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:25.935163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:32801 (local address 127.0.236.65:51329)
0806 15:03:25.935298 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:25.935336 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:25.936218 (+ 882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:25.936470 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:25.936474 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:25.936758 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:25.937011 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:25.937017 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:25.937743 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:25.937744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:25.938602 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:25.938605 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:25.938981 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:25.938982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:25.939115 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:25.939531 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:25.939538 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:25.940810 (+ 1272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:25.942264 (+ 1454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:25.942272 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:25.942274 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:25.942492 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:25.942668 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:25.942669 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:25.942669 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:25.942782 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:25.943013 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:25.943020 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:25.943148 (+ 128us) client_negotiation.cc:769] Sending connection context
0806 15:03:25.943234 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 15:03:25.943380 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":63,"threads_started":1}
I20260806 15:03:25.944046 9460 heartbeater.cc:347] Connected to a master server at 127.0.236.126:32801
I20260806 15:03:25.944140 9460 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:25.944319 9460 heartbeater.cc:511] Master 127.0.236.126:32801 requested a full tablet report, sending...
I20260806 15:03:25.944864 9176 ts_manager.cc:194] Registered new tserver with Master: 659a747346c24036917799688c008ccc (127.0.236.65:41667)
WARNING: no policy specified for kudu/127.0.236.66@KRBTEST.COM; defaulting to no policy
I20260806 15:03:25.945637 9176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.65@KRBTEST.COM'} at 127.0.236.65:51329
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
I20260806 15:03:25.960134 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:0
--local_ip_for_outbound_sockets=127.0.236.66
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:32801
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:35619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:26.038182 9468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:26.038363 9468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:26.038385 9468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:26.039474 9468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:26.039506 9468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:26.039531 9468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:03:26.040941 9468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:35619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:32801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:26.041114 9468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:26.041275 9468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:26.045744 9475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:26.045782 9478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:26.045750 9476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:26.045974 9468 server_base.cc:1034] running on GCE node
I20260806 15:03:26.046221 9468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:26.046470 9468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:26.047622 9468 hybrid_clock.cc:648] HybridClock initialized: now 1786028606047606 us; error 32 us; skew 500 ppm
Aug 06 15:03:26 dist-test-slave-tc2s krb5kdc[9117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028606, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:26.049854 9468 init.cc:377] Logged in from keytab as kudu/127.0.236.66@KRBTEST.COM (short username kudu)
I20260806 15:03:26.050252 9468 webserver.cc:460] Webserver started at http://127.0.236.66:40911/ using document root <none> and password file <none>
I20260806 15:03:26.050447 9468 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:26.050494 9468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:26.050611 9468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:26.051522 9468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-1/data/instance:
uuid: "52b4a4ed05ec4286a33861b9bb6b1195"
format_stamp: "Formatted at 2026-08-06 15:03:26 on dist-test-slave-tc2s"
server_key: "cf5c26ff4516871e9e4c1b8b76faf554"
server_key_iv: "9be09e2879a86e63492b06719fde45d2"
server_key_version: "encryptionkey@0"
I20260806 15:03:26.051823 9468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-1/wal/instance:
uuid: "52b4a4ed05ec4286a33861b9bb6b1195"
format_stamp: "Formatted at 2026-08-06 15:03:26 on dist-test-slave-tc2s"
server_key: "cf5c26ff4516871e9e4c1b8b76faf554"
server_key_iv: "9be09e2879a86e63492b06719fde45d2"
server_key_version: "encryptionkey@0"
I20260806 15:03:26.052958 9468 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:03:26.053544 9487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:26.053689 9468 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:26.053800 9468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "52b4a4ed05ec4286a33861b9bb6b1195"
format_stamp: "Formatted at 2026-08-06 15:03:26 on dist-test-slave-tc2s"
server_key: "cf5c26ff4516871e9e4c1b8b76faf554"
server_key_iv: "9be09e2879a86e63492b06719fde45d2"
server_key_version: "encryptionkey@0"
I20260806 15:03:26.053862 9468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:26.076627 9468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:26.078266 9468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:26.078382 9468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:26.078575 9468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:26.078846 9468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:26.078886 9468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:26.078905 9468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:26.078929 9468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:26.084112 9468 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:39077
I20260806 15:03:26.084178 9650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:39077 every 8 connection(s)
I20260806 15:03:26.084416 9468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
Aug 06 15:03:26 dist-test-slave-tc2s krb5kdc[9117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028606, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:26.092176 9232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:26.085053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:54633 (local address 127.0.236.126:32801)
0806 15:03:26.085102 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:26.085104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:26.085274 (+ 170us) server_negotiation.cc:407] Connection header received
0806 15:03:26.086131 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:26.086132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:26.086141 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:26.086186 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:26.086698 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:26.087188 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:26.088070 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:26.088242 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:26.088283 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:26.090162 (+ 1879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:26.090164 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:26.090164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:26.090171 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:26.091210 (+ 1039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:26.091530 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:26.091530 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:26.091531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:26.091572 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:26.091737 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:26.091738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:26.091738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:26.091827 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:26.091868 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:26.092054 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 15:03:26.092114 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 15:03:26.092334 9655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:26.085029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:32801 (local address 127.0.236.66:54633)
0806 15:03:26.085170 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:26.085202 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:26.086054 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:26.086239 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:26.086243 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:26.086469 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:26.086641 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:26.086645 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:26.087244 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:26.087245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:26.088013 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:26.088015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:26.088291 (+ 276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:26.088292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:26.088377 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:26.088787 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:26.088795 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:26.090074 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:26.091270 (+ 1196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:26.091275 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:26.091277 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:26.091476 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:26.091615 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:26.091616 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:26.091616 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:26.091697 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:26.091867 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:26.091870 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:26.091973 (+ 103us) client_negotiation.cc:769] Sending connection context
0806 15:03:26.092043 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 15:03:26.092149 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
I20260806 15:03:26.092767 9652 heartbeater.cc:347] Connected to a master server at 127.0.236.126:32801
I20260806 15:03:26.092846 9652 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:26.093010 9652 heartbeater.cc:511] Master 127.0.236.126:32801 requested a full tablet report, sending...
I20260806 15:03:26.093335 9176 ts_manager.cc:194] Registered new tserver with Master: 52b4a4ed05ec4286a33861b9bb6b1195 (127.0.236.66:39077)
I20260806 15:03:26.093693 9176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.66@KRBTEST.COM'} at 127.0.236.66:54633
I20260806 15:03:26.094359 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 9468
I20260806 15:03:26.094460 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-1/wal/instance
I20260806 15:03:26.094637 945 external_mini_cluster.cc:1428] Setting key e5760cd56f3cad34b46631a15cd0df7e
WARNING: no policy specified for kudu/127.0.236.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
I20260806 15:03:26.116776 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:0
--local_ip_for_outbound_sockets=127.0.236.67
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:32801
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:35619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:26.194371 9659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:26.194564 9659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:26.194595 9659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:26.195684 9659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:26.195724 9659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:26.195756 9659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:03:26.197170 9659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:35619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:32801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:26.197397 9659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:26.197590 9659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:26.202101 9665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:26.202114 9666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:26.202112 9670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:26.202172 9659 server_base.cc:1034] running on GCE node
I20260806 15:03:26.202461 9659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:26.202685 9659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:26.203810 9659 hybrid_clock.cc:648] HybridClock initialized: now 1786028606203796 us; error 35 us; skew 500 ppm
Aug 06 15:03:26 dist-test-slave-tc2s krb5kdc[9117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028606, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:26.206094 9659 init.cc:377] Logged in from keytab as kudu/127.0.236.67@KRBTEST.COM (short username kudu)
I20260806 15:03:26.206501 9659 webserver.cc:460] Webserver started at http://127.0.236.67:41527/ using document root <none> and password file <none>
I20260806 15:03:26.206707 9659 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:26.206748 9659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:26.206864 9659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:26.207782 9659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-2/data/instance:
uuid: "5fb222925806496a967bde01dad5305e"
format_stamp: "Formatted at 2026-08-06 15:03:26 on dist-test-slave-tc2s"
server_key: "8c8b5cec673deb46fcc1f447fc1fe836"
server_key_iv: "a6177c911a6ea638190d56ec17f873c1"
server_key_version: "encryptionkey@0"
I20260806 15:03:26.208091 9659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-2/wal/instance:
uuid: "5fb222925806496a967bde01dad5305e"
format_stamp: "Formatted at 2026-08-06 15:03:26 on dist-test-slave-tc2s"
server_key: "8c8b5cec673deb46fcc1f447fc1fe836"
server_key_iv: "a6177c911a6ea638190d56ec17f873c1"
server_key_version: "encryptionkey@0"
I20260806 15:03:26.209241 9659 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:03:26.209867 9681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:26.210034 9659 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:26.210119 9659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "5fb222925806496a967bde01dad5305e"
format_stamp: "Formatted at 2026-08-06 15:03:26 on dist-test-slave-tc2s"
server_key: "8c8b5cec673deb46fcc1f447fc1fe836"
server_key_iv: "a6177c911a6ea638190d56ec17f873c1"
server_key_version: "encryptionkey@0"
I20260806 15:03:26.210167 9659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:26.217885 9659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:26.219460 9659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:26.219558 9659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:26.219712 9659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:26.219944 9659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:26.219973 9659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:26.219991 9659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:26.220009 9659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:26.225445 9659 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:42737
I20260806 15:03:26.225486 9846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:42737 every 8 connection(s)
I20260806 15:03:26.225845 9659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
I20260806 15:03:26.230666 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 9659
I20260806 15:03:26.230779 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/minicluster-data/ts-2/wal/instance
I20260806 15:03:26.230940 945 external_mini_cluster.cc:1428] Setting key a6a176c64d17c16cd6ebde6dd635c21c
Aug 06 15:03:26 dist-test-slave-tc2s krb5kdc[9117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028606, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:26.234050 9232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:26.226473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:56595 (local address 127.0.236.126:32801)
0806 15:03:26.226518 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:26.226521 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:26.226711 (+ 190us) server_negotiation.cc:407] Connection header received
0806 15:03:26.227539 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:26.227540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:26.227550 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:26.227594 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:26.228104 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:26.228579 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:26.229441 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:26.229611 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:26.229652 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:26.231585 (+ 1933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:26.231587 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:26.231588 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:26.231595 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:26.232811 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:26.233159 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:26.233161 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:26.233161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:26.233214 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:26.233431 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:26.233432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:26.233432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:26.233538 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:26.233592 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:26.233837 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 15:03:26.233959 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 15:03:26.234157 9849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:26.226457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:32801 (local address 127.0.236.67:56595)
0806 15:03:26.226608 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:26.226639 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:26.227460 (+ 821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:26.227651 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:26.227653 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:26.227880 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:26.228048 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:26.228052 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:26.228636 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:26.228636 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:26.229384 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:26.229385 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:26.229660 (+ 275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:26.229660 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:26.229748 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:26.230162 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:26.230170 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:26.231498 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:26.232884 (+ 1386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:26.232891 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:26.232893 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:26.233094 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:26.233268 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:26.233268 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:26.233269 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:26.233376 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:26.233592 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:26.233598 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:26.233723 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 15:03:26.233806 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 15:03:26.233967 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":90,"threads_started":1}
I20260806 15:03:26.234558 9847 heartbeater.cc:347] Connected to a master server at 127.0.236.126:32801
I20260806 15:03:26.234638 9847 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:26.234793 9847 heartbeater.cc:511] Master 127.0.236.126:32801 requested a full tablet report, sending...
I20260806 15:03:26.235121 9176 ts_manager.cc:194] Registered new tserver with Master: 5fb222925806496a967bde01dad5305e (127.0.236.67:42737)
I20260806 15:03:26.235474 9176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.67@KRBTEST.COM'} at 127.0.236.67:56595
I20260806 15:03:26.242647 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/test-admin.keytab.
Aug 06 15:03:26 dist-test-slave-tc2s krb5kdc[9117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:26.257172 945 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 15:03:26.258669 9859 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:26.258013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41196 (local address 127.0.236.65:41667)
0806 15:03:26.258165 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:26.258170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:26.258180 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:03:26.258206 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:26.258207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:26.258220 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:26.258282 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:26.258584 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41196: BlockingRecv error: recv got EOF from 127.0.0.1:41196 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
W20260806 15:03:26.258843 945 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/krb5cc' of type FILE
Aug 06 15:03:26 dist-test-slave-tc2s krb5kdc[9117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:26.259697 945 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 15:03:26 dist-test-slave-tc2s krb5kdc[9117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:26.260492 945 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 15:03:26 dist-test-slave-tc2s krb5kdc[9117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.65@KRBTEST.COM
I20260806 15:03:26.267470 9859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:26.260935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41200 (local address 127.0.236.65:41667)
0806 15:03:26.260971 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:26.260972 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:26.261060 (+ 88us) server_negotiation.cc:407] Connection header received
0806 15:03:26.261118 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:26.261119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:26.261129 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:26.261172 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:26.261556 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:26.262309 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:26.263023 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:26.263332 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:26.263389 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:26.265184 (+ 1795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:26.265194 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:26.265194 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:26.265200 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:26.266450 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:26.266758 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:26.266761 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:26.266761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:26.266814 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:26.266978 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:26.266978 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:26.266979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:26.267102 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:26.267222 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:26.267300 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 15:03:26.267375 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 15:03:26.268973 9859 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:26.268472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41204 (local address 127.0.236.65:41667)
0806 15:03:26.268510 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:26.268512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:26.268554 (+ 42us) server_negotiation.cc:407] Connection header received
0806 15:03:26.268592 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:26.268593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:26.268601 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:26.268640 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:26.268931 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41204: BlockingRecv error: recv got EOF from 127.0.0.1:41204 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 15:03:26.269094 945 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/krb5cc)
Aug 06 15:03:26 dist-test-slave-tc2s krb5kdc[9117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:26.269929 945 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 15:03:26 dist-test-slave-tc2s krb5kdc[9117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:26.270751 945 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 15:03:26 dist-test-slave-tc2s krb5kdc[9117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.65@KRBTEST.COM
I20260806 15:03:26.276244 9859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:26.271207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41218 (local address 127.0.236.65:41667)
0806 15:03:26.271255 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:26.271257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:26.271264 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:03:26.271299 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:26.271299 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:26.271306 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:26.271344 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:26.271625 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:26.272072 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:26.272654 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:26.272830 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:26.272871 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:26.274537 (+ 1666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:26.274538 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:26.274538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:26.274544 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:26.275516 (+ 972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:26.275776 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:26.275776 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:26.275777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:26.275816 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:26.275935 (+ 119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:26.275935 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:26.275936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:26.276006 (+ 70us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:26.276063 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:26.276123 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 15:03:26.276174 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260806 15:03:26.277534 9859 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:26.277065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41230 (local address 127.0.236.65:41667)
0806 15:03:26.277120 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:26.277122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:26.277132 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:03:26.277188 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:26.277188 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:26.277195 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:26.277234 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:26.277493 (+ 259us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41230: BlockingRecv error: recv got EOF from 127.0.0.1:41230 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260806 15:03:26.277566 945 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028580791223-945-0/krb5kdc/krb5cc' of type FILE
Aug 06 15:03:26 dist-test-slave-tc2s krb5kdc[9117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:26.278441 945 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 15:03:26 dist-test-slave-tc2s krb5kdc[9117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:26.279196 945 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 15:03:26 dist-test-slave-tc2s krb5kdc[9117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.65@KRBTEST.COM
I20260806 15:03:26.284749 9859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:26.279624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41236 (local address 127.0.236.65:41667)
0806 15:03:26.279660 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:26.279661 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:26.279702 (+ 41us) server_negotiation.cc:407] Connection header received
0806 15:03:26.279746 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:26.279747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:26.279753 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:26.279795 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:26.280087 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:26.280526 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:26.281093 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:26.281254 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:26.281293 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:26.283035 (+ 1742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:26.283036 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:26.283036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:26.283041 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:26.284003 (+ 962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:26.284265 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:26.284265 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:26.284266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:26.284305 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:26.284437 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:26.284437 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:26.284437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:26.284508 (+ 71us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:26.284565 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:26.284633 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 15:03:26.284682 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 15:03:26.285239 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 9265
I20260806 15:03:26.304011 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 9468
I20260806 15:03:26.321038 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 9659
I20260806 15:03:26.328536 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 9140
2026-08-06T15:03:26Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1929 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:03:26 dist-test-slave-tc2s krb5kdc[9876](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:03:26 dist-test-slave-tc2s krb5kdc[9876](info): set up 2 sockets
Aug 06 15:03:26 dist-test-slave-tc2s krb5kdc[9876](info): commencing operation
krb5kdc: starting...
W20260806 15:03:27.435143 945 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:03:27 dist-test-slave-tc2s krb5kdc[9876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:03:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:03:27Z Disabled control of system clock
WARNING: no policy specified for oryx/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.126.keytab.
Entry for principal oryx/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.126.keytab.
I20260806 15:03:27.545681 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:35729
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.126.keytab
--principal=oryx/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:39033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:35729
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:27.623821 9893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:27.624008 9893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:27.624038 9893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:27.625123 9893 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:03:27.625162 9893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:27.625180 9893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:27.625197 9893 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:03:27.625211 9893 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:03:27.626657 9893 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:39033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:35729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.126.keytab
--principal=oryx/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:35729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:27.626878 9893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:27.627068 9893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:27.631554 9905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:27.631564 9901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:27.631614 9903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:27.631721 9893 server_base.cc:1034] running on GCE node
I20260806 15:03:27.631975 9893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:27.632282 9893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:27.633419 9893 hybrid_clock.cc:648] HybridClock initialized: now 1786028607633401 us; error 31 us; skew 500 ppm
Aug 06 15:03:27 dist-test-slave-tc2s krb5kdc[9876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028607, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:27.635725 9893 init.cc:377] Logged in from keytab as oryx/127.0.236.126@KRBTEST.COM (short username oryx)
I20260806 15:03:27.636113 9893 webserver.cc:460] Webserver started at http://127.0.236.126:36293/ using document root <none> and password file <none>
I20260806 15:03:27.636298 9893 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:27.636344 9893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:27.636462 9893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:27.637400 9893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "c92fd02274d84e3db9c69443339af692"
format_stamp: "Formatted at 2026-08-06 15:03:27 on dist-test-slave-tc2s"
server_key: "4e9c24577c4e374986222c8c847ab09e"
server_key_iv: "d47c379c266cfb6bf98114e91df3f68a"
server_key_version: "encryptionkey@0"
I20260806 15:03:27.637712 9893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "c92fd02274d84e3db9c69443339af692"
format_stamp: "Formatted at 2026-08-06 15:03:27 on dist-test-slave-tc2s"
server_key: "4e9c24577c4e374986222c8c847ab09e"
server_key_iv: "d47c379c266cfb6bf98114e91df3f68a"
server_key_version: "encryptionkey@0"
I20260806 15:03:27.638950 9893 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:03:27.639631 9917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:27.639770 9893 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:27.639853 9893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "c92fd02274d84e3db9c69443339af692"
format_stamp: "Formatted at 2026-08-06 15:03:27 on dist-test-slave-tc2s"
server_key: "4e9c24577c4e374986222c8c847ab09e"
server_key_iv: "d47c379c266cfb6bf98114e91df3f68a"
server_key_version: "encryptionkey@0"
I20260806 15:03:27.639917 9893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:27.652230 9893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:27.654184 9893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:27.654313 9893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:27.658490 9893 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:35729
I20260806 15:03:27.658608 9991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:35729 every 8 connection(s)
I20260806 15:03:27.658852 9893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:03:27.659667 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 9893
I20260806 15:03:27.659790 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:03:27.659977 945 external_mini_cluster.cc:1428] Setting key 64b60e7d56641d63ac0806a6ae509ab4
I20260806 15:03:27.659976 9992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:27.662570 9992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692: Bootstrap starting.
I20260806 15:03:27.663128 9992 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:27.663372 9992 log.cc:824] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:27.663991 9992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692: No bootstrap required, opened a new log
I20260806 15:03:27.665848 9992 raft_consensus.cc:348] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } }
I20260806 15:03:27.666054 9992 raft_consensus.cc:374] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:27.666098 9992 raft_consensus.cc:729] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c92fd02274d84e3db9c69443339af692, State: Initialized, Role: FOLLOWER
Aug 06 15:03:27 dist-test-slave-tc2s krb5kdc[9876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM
I20260806 15:03:27.666216 9992 consensus_queue.cc:260] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } }
I20260806 15:03:27.666296 9992 raft_consensus.cc:388] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:03:27.666329 9992 raft_consensus.cc:482] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:03:27.666378 9992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:27.666838 9992 raft_consensus.cc:504] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } }
I20260806 15:03:27.666942 9992 leader_election.cc:302] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c92fd02274d84e3db9c69443339af692; no voters:
I20260806 15:03:27.667089 9992 leader_election.cc:288] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:03:27.667142 9998 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:27.667279 9998 raft_consensus.cc:686] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [term 1 LEADER]: Becoming Leader. State: Replica: c92fd02274d84e3db9c69443339af692, State: Running, Role: LEADER
I20260806 15:03:27.667330 9992 sys_catalog.cc:564] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:03:27.667374 9998 consensus_queue.cc:237] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } }
I20260806 15:03:27.667693 9998 sys_catalog.cc:455] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c92fd02274d84e3db9c69443339af692. Latest consensus state: current_term: 1 leader_uuid: "c92fd02274d84e3db9c69443339af692" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } }
I20260806 15:03:27.667773 9998 sys_catalog.cc:458] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:27.667742 9999 sys_catalog.cc:455] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c92fd02274d84e3db9c69443339af692" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } }
I20260806 15:03:27.668116 9999 sys_catalog.cc:458] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:27.668560 10014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:03:27.669049 10014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:03:27.670018 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:27.660464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42400 (local address 127.0.236.126:35729)
0806 15:03:27.660653 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:27.660660 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:27.660683 (+ 23us) server_negotiation.cc:407] Connection header received
0806 15:03:27.661230 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:27.661236 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:27.661280 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:27.661397 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:27.662010 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:27.662932 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:27.663808 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:27.664228 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:27.664322 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:27.666697 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:27.666702 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:27.666702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:27.666713 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:27.668544 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:27.668935 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:27.668939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:27.668939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:27.669025 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:27.669204 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:27.669205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:27.669205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:27.669432 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:27.669568 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:27.669668 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 15:03:27.669794 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":89,"threads_started":1}
I20260806 15:03:27.670549 10014 catalog_manager.cc:1347] Generated new cluster ID: fef1eb67d42c483abc65fd8ff931c254
I20260806 15:03:27.670595 10014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:03:27.712626 10014 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:03:27.713148 10014 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:03:27.720127 10014 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692: Generated new TSK 0
I20260806 15:03:27.720281 10014 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.0.236.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.236.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.236.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.65.keytab.
Entry for principal oryx/127.0.236.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.65.keytab.
I20260806 15:03:27.758790 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:0
--local_ip_for_outbound_sockets=127.0.236.65
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:35729
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.65.keytab
--principal=oryx/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:39033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:27.918390 10032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:27.918644 10032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:27.918673 10032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:27.920205 10032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:27.920250 10032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:27.920279 10032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:03:27.922394 10032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:39033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.65.keytab
--principal=oryx/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:35729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:27.922652 10032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:27.922870 10032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:27.927625 10038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:27.927636 10041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:27.927636 10039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:27.927827 10032 server_base.cc:1034] running on GCE node
I20260806 15:03:27.928072 10032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:27.928315 10032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:27.929471 10032 hybrid_clock.cc:648] HybridClock initialized: now 1786028607929450 us; error 35 us; skew 500 ppm
Aug 06 15:03:27 dist-test-slave-tc2s krb5kdc[9876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028607, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:27.931885 10032 init.cc:377] Logged in from keytab as oryx/127.0.236.65@KRBTEST.COM (short username oryx)
I20260806 15:03:27.932263 10032 webserver.cc:460] Webserver started at http://127.0.236.65:39667/ using document root <none> and password file <none>
I20260806 15:03:27.932480 10032 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:27.932524 10032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:27.932643 10032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:27.933634 10032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/data/instance:
uuid: "120512845c1a446db9c5b628af55f0db"
format_stamp: "Formatted at 2026-08-06 15:03:27 on dist-test-slave-tc2s"
server_key: "9f778e6dd57585b66a834892494d73d0"
server_key_iv: "f670c7fcbacd15cc15f8256e84325eb3"
server_key_version: "encryptionkey@0"
I20260806 15:03:27.933992 10032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/wal/instance:
uuid: "120512845c1a446db9c5b628af55f0db"
format_stamp: "Formatted at 2026-08-06 15:03:27 on dist-test-slave-tc2s"
server_key: "9f778e6dd57585b66a834892494d73d0"
server_key_iv: "f670c7fcbacd15cc15f8256e84325eb3"
server_key_version: "encryptionkey@0"
I20260806 15:03:27.935182 10032 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:27.935789 10052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:27.935943 10032 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:27.936031 10032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "120512845c1a446db9c5b628af55f0db"
format_stamp: "Formatted at 2026-08-06 15:03:27 on dist-test-slave-tc2s"
server_key: "9f778e6dd57585b66a834892494d73d0"
server_key_iv: "f670c7fcbacd15cc15f8256e84325eb3"
server_key_version: "encryptionkey@0"
I20260806 15:03:27.936087 10032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:27.949437 10032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:27.951637 10032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:27.951740 10032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:27.951910 10032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:27.952150 10032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:27.952181 10032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:27.952200 10032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:27.952211 10032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:27.957528 10032 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:36187
I20260806 15:03:27.957569 10208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:36187 every 8 connection(s)
I20260806 15:03:27.957835 10032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
Aug 06 15:03:27 dist-test-slave-tc2s krb5kdc[9876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028607, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM
I20260806 15:03:27.964524 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 10032
I20260806 15:03:27.964628 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/wal/instance
I20260806 15:03:27.964829 945 external_mini_cluster.cc:1428] Setting key b55da447ff5faf9c40a962b8636759fa
I20260806 15:03:27.966579 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:27.958475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:46709 (local address 127.0.236.126:35729)
0806 15:03:27.958541 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:27.958544 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:27.958707 (+ 163us) server_negotiation.cc:407] Connection header received
0806 15:03:27.959285 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:27.959287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:27.959299 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:27.959375 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:27.960275 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:27.960784 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:27.961688 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:27.961884 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:27.961960 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:27.963765 (+ 1805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:27.963767 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:27.963767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:27.963778 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:27.965308 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:27.965701 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:27.965702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:27.965703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:27.965760 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:27.965981 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:27.965982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:27.965982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:27.966109 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:27.966177 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:27.966412 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 15:03:27.966496 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 15:03:27.966696 10211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:27.958435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.65:46709)
0806 15:03:27.958595 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:27.958633 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:27.959189 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:27.959449 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:27.959453 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:27.960011 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:27.960191 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:27.960195 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:27.960847 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:27.960848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:27.961625 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:27.961627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:27.961964 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:27.961964 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:27.962059 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:27.962420 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:27.962428 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:27.963679 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:27.965371 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:27.965379 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:27.965380 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:27.965634 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:27.965819 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:27.965820 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:27.965820 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:27.965934 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:27.966168 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:27.966171 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:27.966275 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 15:03:27.966347 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 15:03:27.966474 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":92,"threads_started":1}
I20260806 15:03:27.967141 10209 heartbeater.cc:347] Connected to a master server at 127.0.236.126:35729
I20260806 15:03:27.967231 10209 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:27.967391 10209 heartbeater.cc:511] Master 127.0.236.126:35729 requested a full tablet report, sending...
I20260806 15:03:27.967950 9945 ts_manager.cc:194] Registered new tserver with Master: 120512845c1a446db9c5b628af55f0db (127.0.236.65:36187)
I20260806 15:03:27.968560 9945 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.236.65@KRBTEST.COM'} at 127.0.236.65:46709
WARNING: no policy specified for oryx/127.0.236.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.236.66@KRBTEST.COM" created.
I20260806 15:03:28.969417 10209 heartbeater.cc:503] Master 127.0.236.126:35729 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.236.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.66.keytab.
Entry for principal oryx/127.0.236.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.66.keytab.
W20260806 15:03:28.970510 945 mini_kdc.cc:262] Time spent creating service keytab for oryx/127.0.236.66: real 1.005s user 0.000s sys 0.003s
I20260806 15:03:28.970818 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:0
--local_ip_for_outbound_sockets=127.0.236.66
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:35729
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.66.keytab
--principal=oryx/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:39033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:29.049187 10215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:29.049364 10215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:29.049386 10215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:29.050442 10215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:29.050477 10215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:29.050500 10215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:03:29.051862 10215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:39033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.66.keytab
--principal=oryx/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:35729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:29.052037 10215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:29.052201 10215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:29.056761 10221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:29.056761 10224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:29.056775 10222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:29.057114 10215 server_base.cc:1034] running on GCE node
I20260806 15:03:29.057312 10215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:29.057544 10215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:29.058686 10215 hybrid_clock.cc:648] HybridClock initialized: now 1786028609058666 us; error 29 us; skew 500 ppm
Aug 06 15:03:29 dist-test-slave-tc2s krb5kdc[9876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028609, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:29.060979 10215 init.cc:377] Logged in from keytab as oryx/127.0.236.66@KRBTEST.COM (short username oryx)
I20260806 15:03:29.061390 10215 webserver.cc:460] Webserver started at http://127.0.236.66:40617/ using document root <none> and password file <none>
I20260806 15:03:29.061602 10215 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:29.061650 10215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:29.061772 10215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:29.062762 10215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/data/instance:
uuid: "b7ccff12a0d44255a60b61e0e3287c5c"
format_stamp: "Formatted at 2026-08-06 15:03:29 on dist-test-slave-tc2s"
server_key: "2254db597bfcf0e41be383f40597efe3"
server_key_iv: "eab5433e09ad47f25695507b41106ac2"
server_key_version: "encryptionkey@0"
I20260806 15:03:29.063079 10215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/wal/instance:
uuid: "b7ccff12a0d44255a60b61e0e3287c5c"
format_stamp: "Formatted at 2026-08-06 15:03:29 on dist-test-slave-tc2s"
server_key: "2254db597bfcf0e41be383f40597efe3"
server_key_iv: "eab5433e09ad47f25695507b41106ac2"
server_key_version: "encryptionkey@0"
I20260806 15:03:29.064234 10215 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:03:29.064836 10232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:29.064980 10215 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:29.065061 10215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "b7ccff12a0d44255a60b61e0e3287c5c"
format_stamp: "Formatted at 2026-08-06 15:03:29 on dist-test-slave-tc2s"
server_key: "2254db597bfcf0e41be383f40597efe3"
server_key_iv: "eab5433e09ad47f25695507b41106ac2"
server_key_version: "encryptionkey@0"
I20260806 15:03:29.065126 10215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:29.075126 10215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:29.076701 10215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:29.076828 10215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:29.077036 10215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:29.077306 10215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:29.077339 10215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:29.077368 10215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:29.077384 10215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:29.082773 10215 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:34923
I20260806 15:03:29.082813 10403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:34923 every 8 connection(s)
I20260806 15:03:29.083086 10215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
I20260806 15:03:29.085570 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 10215
I20260806 15:03:29.085664 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/wal/instance
I20260806 15:03:29.085842 945 external_mini_cluster.cc:1428] Setting key 087ef17351d6dace31c9a9de2fbdc5c9
Aug 06 15:03:29 dist-test-slave-tc2s krb5kdc[9876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028609, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM
I20260806 15:03:29.092657 10408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:29.083779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:55853 (local address 127.0.236.126:35729)
0806 15:03:29.084424 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:29.084426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:29.084432 (+ 6us) server_negotiation.cc:407] Connection header received
0806 15:03:29.084453 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:29.084454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:29.084463 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:29.084518 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:29.085407 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.086003 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.087297 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.087493 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.087543 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:29.089796 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:29.089798 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:29.089798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:29.089806 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:29.091372 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:29.091756 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:29.091758 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:29.091759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:29.091831 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:29.092041 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:29.092042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:29.092042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:29.092184 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:29.092254 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:29.092455 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 15:03:29.092546 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":51,"threads_started":1}
I20260806 15:03:29.092674 10406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:29.083698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.66:55853)
0806 15:03:29.083843 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:29.083882 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:29.084294 (+ 412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:29.084612 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:29.084617 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:29.085143 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:29.085313 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:29.085318 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.086095 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.086097 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:29.087219 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:29.087222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.087564 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.087565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:29.087693 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:29.088211 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:29.088221 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:29.089708 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:29.091468 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:29.091474 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:29.091476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:29.091682 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:29.091894 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:29.091895 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:29.091895 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:29.091986 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:29.092240 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:29.092243 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:29.092343 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 15:03:29.092415 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 15:03:29.092522 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260806 15:03:29.093165 10404 heartbeater.cc:347] Connected to a master server at 127.0.236.126:35729
I20260806 15:03:29.093250 10404 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:29.093415 10404 heartbeater.cc:511] Master 127.0.236.126:35729 requested a full tablet report, sending...
I20260806 15:03:29.093770 9945 ts_manager.cc:194] Registered new tserver with Master: b7ccff12a0d44255a60b61e0e3287c5c (127.0.236.66:34923)
I20260806 15:03:29.094157 9945 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.236.66@KRBTEST.COM'} at 127.0.236.66:55853
WARNING: no policy specified for oryx/127.0.236.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.236.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.236.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.67.keytab.
Entry for principal oryx/127.0.236.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.67.keytab.
W20260806 15:03:29.617079 945 mini_kdc.cc:262] Time spent creating service keytab for oryx/127.0.236.67: real 0.530s user 0.002s sys 0.002s
I20260806 15:03:29.617408 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:0
--local_ip_for_outbound_sockets=127.0.236.67
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:35729
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.67.keytab
--principal=oryx/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:39033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:29.698468 10411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:29.698655 10411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:29.698688 10411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:29.699779 10411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:29.699819 10411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:29.699853 10411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:03:29.701270 10411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:39033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.67.keytab
--principal=oryx/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:35729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:29.701505 10411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:29.701702 10411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:29.706274 10417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:29.706295 10411 server_base.cc:1034] running on GCE node
W20260806 15:03:29.706422 10418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:29.706272 10420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:29.706660 10411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:29.706900 10411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:29.708045 10411 hybrid_clock.cc:648] HybridClock initialized: now 1786028609708030 us; error 27 us; skew 500 ppm
Aug 06 15:03:29 dist-test-slave-tc2s krb5kdc[9876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028609, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:29.710433 10411 init.cc:377] Logged in from keytab as oryx/127.0.236.67@KRBTEST.COM (short username oryx)
I20260806 15:03:29.710840 10411 webserver.cc:460] Webserver started at http://127.0.236.67:38517/ using document root <none> and password file <none>
I20260806 15:03:29.711040 10411 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:29.711087 10411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:29.711208 10411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:29.712139 10411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/data/instance:
uuid: "8194526e9ac244b7b939dadfa140b2c7"
format_stamp: "Formatted at 2026-08-06 15:03:29 on dist-test-slave-tc2s"
server_key: "7c0c6c6b4918d6a040582d88f0cda804"
server_key_iv: "dcdb113b584426a143628d4f2ec99c4f"
server_key_version: "encryptionkey@0"
I20260806 15:03:29.712461 10411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/wal/instance:
uuid: "8194526e9ac244b7b939dadfa140b2c7"
format_stamp: "Formatted at 2026-08-06 15:03:29 on dist-test-slave-tc2s"
server_key: "7c0c6c6b4918d6a040582d88f0cda804"
server_key_iv: "dcdb113b584426a143628d4f2ec99c4f"
server_key_version: "encryptionkey@0"
I20260806 15:03:29.713738 10411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:03:29.714424 10433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:29.714581 10411 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:29.714661 10411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "8194526e9ac244b7b939dadfa140b2c7"
format_stamp: "Formatted at 2026-08-06 15:03:29 on dist-test-slave-tc2s"
server_key: "7c0c6c6b4918d6a040582d88f0cda804"
server_key_iv: "dcdb113b584426a143628d4f2ec99c4f"
server_key_version: "encryptionkey@0"
I20260806 15:03:29.714730 10411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:29.734942 10411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:29.736922 10411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:29.737047 10411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:29.737249 10411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:29.737525 10411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:29.737565 10411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:29.737593 10411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:29.737617 10411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:29.743052 10411 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:35007
I20260806 15:03:29.743109 10585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:35007 every 8 connection(s)
I20260806 15:03:29.743405 10411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
Aug 06 15:03:29 dist-test-slave-tc2s krb5kdc[9876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028609, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM
I20260806 15:03:29.751847 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:29.744065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:38825 (local address 127.0.236.126:35729)
0806 15:03:29.744196 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:29.744198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:29.744317 (+ 119us) server_negotiation.cc:407] Connection header received
0806 15:03:29.744795 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:29.744797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:29.744811 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:29.744868 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:29.745778 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.746329 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.747289 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.747503 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.747560 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:29.749406 (+ 1846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:29.749408 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:29.749408 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:29.749416 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:29.750593 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:29.750972 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:29.750973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:29.750973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:29.751031 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:29.751243 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:29.751244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:29.751244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:29.751351 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:29.751407 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:29.751661 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 15:03:29.751751 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
I20260806 15:03:29.751968 10588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:29.744032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.67:38825)
0806 15:03:29.744199 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:29.744241 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:29.744700 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:29.744945 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:29.744949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:29.745509 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:29.745685 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:29.745689 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.746419 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.746420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:29.747207 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:29.747209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.747573 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.747574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:29.747692 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:29.748056 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:29.748064 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:29.749316 (+ 1252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:29.750664 (+ 1348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:29.750671 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:29.750673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:29.750880 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:29.751087 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:29.751088 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:29.751088 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:29.751186 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:29.751404 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:29.751411 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:29.751545 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 15:03:29.751630 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 15:03:29.751772 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260806 15:03:29.752324 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 10411
I20260806 15:03:29.752442 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/wal/instance
I20260806 15:03:29.752478 10586 heartbeater.cc:347] Connected to a master server at 127.0.236.126:35729
I20260806 15:03:29.752575 10586 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:29.752637 945 external_mini_cluster.cc:1428] Setting key 562646416332fc8a6a7207a2dae7822e
I20260806 15:03:29.752748 10586 heartbeater.cc:511] Master 127.0.236.126:35729 requested a full tablet report, sending...
I20260806 15:03:29.753151 9945 ts_manager.cc:194] Registered new tserver with Master: 8194526e9ac244b7b939dadfa140b2c7 (127.0.236.67:35007)
I20260806 15:03:29.753484 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:03:29.753600 9945 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.236.67@KRBTEST.COM'} at 127.0.236.67:38825
WARNING: no policy specified for oryx/127.0.236.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.236.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.236.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.125.keytab.
Entry for principal oryx/127.0.236.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.125.keytab.
I20260806 15:03:29.776238 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.125:38713
--webserver_interface=127.0.236.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.125.keytab
--principal=oryx/127.0.236.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:39033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:35729,127.0.236.125:38713
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:29.853461 10594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:29.853636 10594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:29.853654 10594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:29.854758 10594 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:03:29.854795 10594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:29.854808 10594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:29.854822 10594 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:03:29.854846 10594 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:03:29.856261 10594 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:39033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:35729,127.0.236.125:38713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.125.keytab
--principal=oryx/127.0.236.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.125:38713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.236.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
W20260806 15:03:29.856359 10594 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.0.236.126:35729,127.0.236.125:38713'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 15:03:29.856489 10594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:29.856675 10594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:29.861054 10601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:29.861044 10600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:29.861047 10603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:29.861514 10594 server_base.cc:1034] running on GCE node
I20260806 15:03:29.861747 10594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:29.862021 10594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:29.863158 10594 hybrid_clock.cc:648] HybridClock initialized: now 1786028609863145 us; error 23 us; skew 500 ppm
Aug 06 15:03:29 dist-test-slave-tc2s krb5kdc[9876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028609, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:29.865403 10594 init.cc:377] Logged in from keytab as oryx/127.0.236.125@KRBTEST.COM (short username oryx)
I20260806 15:03:29.865796 10594 webserver.cc:460] Webserver started at http://127.0.236.125:39503/ using document root <none> and password file <none>
I20260806 15:03:29.866030 10594 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:29.866074 10594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:29.866189 10594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:29.867308 10594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-1/data/instance:
uuid: "e7d2710f37144ab1bd06312edcde3f45"
format_stamp: "Formatted at 2026-08-06 15:03:29 on dist-test-slave-tc2s"
server_key: "fc1f004dc67141add0fc5782b8372808"
server_key_iv: "0511ea2668ede40d4cf5685f11940ad0"
server_key_version: "encryptionkey@0"
I20260806 15:03:29.867691 10594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-1/wal/instance:
uuid: "e7d2710f37144ab1bd06312edcde3f45"
format_stamp: "Formatted at 2026-08-06 15:03:29 on dist-test-slave-tc2s"
server_key: "fc1f004dc67141add0fc5782b8372808"
server_key_iv: "0511ea2668ede40d4cf5685f11940ad0"
server_key_version: "encryptionkey@0"
I20260806 15:03:29.869062 10594 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:29.869701 10610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:29.869871 10594 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:29.869982 10594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-1/wal
uuid: "e7d2710f37144ab1bd06312edcde3f45"
format_stamp: "Formatted at 2026-08-06 15:03:29 on dist-test-slave-tc2s"
server_key: "fc1f004dc67141add0fc5782b8372808"
server_key_iv: "0511ea2668ede40d4cf5685f11940ad0"
server_key_version: "encryptionkey@0"
I20260806 15:03:29.870047 10594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:29.913314 10594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:29.915021 10594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:29.915140 10594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:29.918991 10594 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.125:38713
I20260806 15:03:29.919026 10674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.125:38713 every 8 connection(s)
I20260806 15:03:29.919324 10594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-1/data/info.pb
I20260806 15:03:29.920240 10675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:29.920469 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 10594
I20260806 15:03:29.922353 10675 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 15:03:29 dist-test-slave-tc2s krb5kdc[9876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.236.125@KRBTEST.COM
Aug 06 15:03:29 dist-test-slave-tc2s krb5kdc[9876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028609, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.125@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM
I20260806 15:03:29.930245 10679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:29.921035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42206 (local address 127.0.236.125:38713)
0806 15:03:29.921190 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:29.921197 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:29.921221 (+ 24us) server_negotiation.cc:407] Connection header received
0806 15:03:29.921622 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:29.921628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:29.921671 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:29.921776 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:29.922550 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.923592 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.924315 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.924699 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.924779 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:29.927254 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:29.927260 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:29.927260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:29.927266 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:29.928812 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:29.929169 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:29.929171 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:29.929171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:29.929265 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:29.929441 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:29.929441 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:29.929442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:29.929697 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:29.929811 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:29.929970 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 15:03:29.930056 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":67,"threads_started":1}
I20260806 15:03:29.930364 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:29.923049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42402 (local address 127.0.236.126:35729)
0806 15:03:29.923106 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:29.923108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:29.923226 (+ 118us) server_negotiation.cc:407] Connection header received
0806 15:03:29.923278 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:29.923279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:29.923291 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:29.923344 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:29.923744 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.924217 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.925058 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.925347 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.925399 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:29.927821 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:29.927822 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:29.927822 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:29.927830 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:29.929140 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:29.929488 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:29.929489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:29.929489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:29.929539 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:29.929703 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:29.929703 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:29.929704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:29.929876 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:29.929963 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:29.930183 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 15:03:29.930266 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 15:03:29.930374 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:29.922977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.0.1:42402)
0806 15:03:29.923134 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:29.923169 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:29.923237 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:29.923368 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:29.923370 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:29.923599 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:29.923670 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:29.923672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.924309 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.924309 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:29.924981 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:29.924984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.925433 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.925434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:29.925508 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:29.925876 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:29.925886 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:29.927741 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:29.929206 (+ 1465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:29.929212 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:29.929214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:29.929425 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:29.929591 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:29.929591 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:29.929592 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:29.929648 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:29.929993 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:29.930000 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:29.930052 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 15:03:29.930136 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 15:03:29.930246 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":85,"threads_started":1}
I20260806 15:03:29.931034 10675 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 15:03:29 dist-test-slave-tc2s krb5kdc[9876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028609, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.125@KRBTEST.COM for oryx/127.0.236.125@KRBTEST.COM
I20260806 15:03:29.936180 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:29.931239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:38713 (local address 127.0.0.1:42218)
0806 15:03:29.931277 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:29.931287 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:29.931319 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:29.931415 (+ 96us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:29.931416 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:29.931525 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:29.931552 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:29.931553 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.932131 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.932131 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:29.932583 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:29.932584 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.932855 (+ 271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.932855 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:29.932903 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:29.933184 (+ 281us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:29.933188 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:29.934256 (+ 1068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:29.935523 (+ 1267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:29.935524 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:29.935524 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:29.935660 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:29.935785 (+ 125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:29.935786 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:29.935786 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:29.935824 (+ 38us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:29.935971 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:29.935972 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:29.936002 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 15:03:29.936044 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 15:03:29.936110 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 15:03:29.936240 10679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:29.931268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42218 (local address 127.0.236.125:38713)
0806 15:03:29.931309 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:29.931311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:29.931319 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:29.931365 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:29.931366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:29.931372 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:29.931417 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:29.931589 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.932071 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.932639 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.932812 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.932851 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:29.934331 (+ 1480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:29.934331 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:29.934332 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:29.934337 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:29.935466 (+ 1129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:29.935710 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:29.935710 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:29.935711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:29.935749 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:29.935854 (+ 105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:29.935854 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:29.935855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:29.935933 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:29.935974 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:29.936060 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 15:03:29.936148 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":25}
I20260806 15:03:29.937211 10675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45: Bootstrap starting.
I20260806 15:03:29.937939 10675 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:29.938192 10675 log.cc:824] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:29.938833 10675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45: No bootstrap required, opened a new log
I20260806 15:03:29.940083 10675 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } }
I20260806 15:03:29.940184 10675 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:29.940214 10675 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7d2710f37144ab1bd06312edcde3f45, State: Initialized, Role: FOLLOWER
I20260806 15:03:29.940333 10675 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } }
I20260806 15:03:29.940608 10675 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:03:29.940568 10683 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } } }
I20260806 15:03:29.940665 10683 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 15:03:29.942080 10696 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:03:29.942133 10696 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:03:29.943527 10594 master_runner.cc:428] Detected that this master e7d2710f37144ab1bd06312edcde3f45 is joining an existing cluster
I20260806 15:03:29.943596 10594 master_runner.cc:432] Initiating AddMaster RPC to add 127.0.236.125:38713
I20260806 15:03:29.951465 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:29.944515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42412 (local address 127.0.236.126:35729)
0806 15:03:29.944552 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:29.944554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:29.944695 (+ 141us) server_negotiation.cc:407] Connection header received
0806 15:03:29.944755 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:29.944757 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:29.944765 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:29.944808 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:29.945128 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.945841 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.946754 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.947054 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.947118 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:29.947977 (+ 859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:29.947979 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:29.947979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:29.947986 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:29.949346 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:29.950759 (+ 1413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:29.950760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:29.950761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:29.950816 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:29.950973 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:29.950974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:29.950974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:29.951096 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:29.951159 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:29.951292 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 15:03:29.951367 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:03:29.951478 10707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:29.944515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.0.1:42412)
0806 15:03:29.944641 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:29.944650 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:29.944704 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:29.944822 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:29.944822 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:29.944984 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:29.945064 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:29.945066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.945952 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.945953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:29.946672 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:29.946674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.947143 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.947144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:29.947201 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:29.947588 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:29.947593 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:29.947904 (+ 311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:29.949691 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:29.949692 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:29.949692 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:29.950694 (+ 1002us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:29.950865 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:29.950866 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:29.950866 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:29.950922 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:29.951159 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:29.951160 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:29.951200 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 15:03:29.951264 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 15:03:29.951369 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
I20260806 15:03:29.951529 10679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:29.945021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42228 (local address 127.0.236.125:38713)
0806 15:03:29.945061 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:29.945062 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:29.945930 (+ 868us) server_negotiation.cc:407] Connection header received
0806 15:03:29.945947 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:29.945947 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:29.945954 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:29.946003 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:29.946331 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.946903 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.947749 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.948035 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.948101 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:29.948967 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:29.948968 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:29.948969 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:29.948975 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:29.950421 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:29.950797 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:29.950798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:29.950798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:29.950854 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:29.951045 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:29.951046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:29.951046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:29.951151 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:29.951206 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:29.951335 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 15:03:29.951418 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":23}
I20260806 15:03:29.951741 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:29.944871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:38713 (local address 127.0.0.1:42228)
0806 15:03:29.945059 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:29.945068 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:29.945114 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:29.946014 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:29.946015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:29.946207 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:29.946259 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:29.946261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.946976 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.946976 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:29.947667 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:29.947670 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.948119 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.948120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:29.948196 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:29.948553 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:29.948558 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:29.948887 (+ 329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:29.950494 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:29.950495 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:29.950495 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:29.950730 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:29.950923 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:29.950924 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:29.950924 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:29.950999 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:29.951200 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:29.951201 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:29.951237 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 15:03:29.951358 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 15:03:29.951643 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":79,"threads_started":1}
Aug 06 15:03:29 dist-test-slave-tc2s krb5kdc[9876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028607, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.126@KRBTEST.COM for oryx/127.0.236.125@KRBTEST.COM
I20260806 15:03:29.960060 10679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:29.953867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42236 (local address 127.0.236.125:38713)
0806 15:03:29.953935 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:29.953943 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:29.954059 (+ 116us) server_negotiation.cc:407] Connection header received
0806 15:03:29.954128 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:29.954129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:29.954137 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:29.954181 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:29.954557 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.954980 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.955679 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.955865 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.955914 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:29.957827 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:29.957828 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:29.957828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:29.957834 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:29.959025 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:29.959387 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:29.959389 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:29.959389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:29.959441 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:29.959601 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:29.959602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:29.959602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:29.959697 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:29.959748 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:29.959903 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 15:03:29.959978 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 15:03:29.960074 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:29.953826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:38713 (local address 127.0.0.1:42236)
0806 15:03:29.953967 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:29.954003 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:29.954069 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:29.954184 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:29.954187 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:29.954434 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:29.954487 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:29.954489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.955045 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.955046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:29.955608 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:29.955610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.955920 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.955921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:29.955996 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:29.956378 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:29.956383 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:29.957753 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:29.959097 (+ 1344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:29.959104 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:29.959107 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:29.959321 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:29.959494 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:29.959495 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:29.959495 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:29.959552 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:29.959747 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:29.959755 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:29.959805 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 15:03:29.959874 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 15:03:29.959981 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260806 15:03:29.960645 9945 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.0.236.125:38713: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45"
member_type: NON_VOTER
last_known_addr {
host: "127.0.236.125"
port: 38713
}
attrs {
promote: true
}
}
dest_uuid: "c92fd02274d84e3db9c69443339af692"
cas_config_opid_index: -1
I20260806 15:03:29.960947 9945 consensus_queue.cc:237] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: NON_VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: true } }
I20260806 15:03:29.961443 10712 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER e7d2710f37144ab1bd06312edcde3f45 (127.0.236.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: NON_VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: true } } }
I20260806 15:03:29.961684 10641 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:29.962443 10641 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 1 FOLLOWER]: Refusing update from remote peer c92fd02274d84e3db9c69443339af692: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 15:03:29.962456 10712 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.0.236.125:38713
I20260806 15:03:29.962535 10712 sys_catalog.cc:455] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c92fd02274d84e3db9c69443339af692" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: NON_VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: true } } }
I20260806 15:03:29.962591 10712 sys_catalog.cc:458] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:29.962785 10594 master.cc:537] Master@127.0.236.125:38713 shutting down...
I20260806 15:03:29.962812 10714 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: NON_VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 15:03:29.963493 9918 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260806 15:03:29.963925 9918 consensus_peers.cc:487] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 -> Peer e7d2710f37144ab1bd06312edcde3f45 (127.0.236.125:38713): Couldn't send request to peer e7d2710f37144ab1bd06312edcde3f45. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 15:03:29.965327 10594 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:29.965404 10594 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:29.965451 10594 tablet_replica.cc:330] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45: stopping tablet replica
I20260806 15:03:29.978273 10594 master.cc:552] Master@127.0.236.125:38713 shutdown complete.
I20260806 15:03:29.978309 10594 master_runner.cc:305] Clearing existing system tablet
I20260806 15:03:29.978416 10594 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:29.978457 10594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:29.978955 10594 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:03:29.979360 10720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:29.979475 10594 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:29.979544 10594 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:29.979569 10594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-1/wal
uuid: "e7d2710f37144ab1bd06312edcde3f45"
format_stamp: "Formatted at 2026-08-06 15:03:29 on dist-test-slave-tc2s"
server_key: "fc1f004dc67141add0fc5782b8372808"
server_key_iv: "0511ea2668ede40d4cf5685f11940ad0"
server_key_version: "encryptionkey@0"
I20260806 15:03:29.980412 10594 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:29.981757 10594 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 15:03:29.981823 10594 log.cc:1196] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 15:03:29.982051 10594 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45: Deleting consensus metadata
I20260806 15:03:29.982169 10594 master_runner.cc:315] Copying system tablet from 127.0.236.126:35729
I20260806 15:03:29.982717 10594 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45: tablet copy: Beginning tablet copy session from remote peer at address 127.0.236.126:35729
I20260806 15:03:29.988291 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:29.982975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.0.1:42420)
0806 15:03:29.983072 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:29.983082 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:29.983141 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:29.983248 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:29.983248 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:29.983426 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:29.983488 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:29.983490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.984071 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.984072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:29.984939 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:29.984942 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.985225 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.985226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:29.985292 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:29.985657 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:29.985661 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:29.986044 (+ 383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:29.987357 (+ 1313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:29.987358 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:29.987359 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:29.987569 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:29.987724 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:29.987724 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:29.987725 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:29.987784 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:29.987974 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:29.987975 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:29.988017 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 15:03:29.988096 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 15:03:29.988190 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":58,"threads_started":1}
I20260806 15:03:29.988293 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:29.983046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42420 (local address 127.0.236.126:35729)
0806 15:03:29.983087 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:29.983089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:29.983128 (+ 39us) server_negotiation.cc:407] Connection header received
0806 15:03:29.983192 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:29.983192 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:29.983200 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:29.983242 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:29.983546 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.984007 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.985008 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:29.985172 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:29.985210 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:29.986116 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:29.986117 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:29.986117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:29.986123 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:29.987292 (+ 1169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:29.987631 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:29.987632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:29.987632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:29.987676 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:29.987835 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:29.987835 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:29.987835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:29.987921 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:29.987970 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:29.988141 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 15:03:29.988213 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 15:03:29.989013 9974 tablet_copy_service.cc:140] P c92fd02274d84e3db9c69443339af692: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e7d2710f37144ab1bd06312edcde3f45 ({username='oryx', principal='oryx/127.0.236.125@KRBTEST.COM'} at 127.0.0.1:42420)
I20260806 15:03:29.989116 9974 tablet_copy_service.cc:161] P c92fd02274d84e3db9c69443339af692: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e7d2710f37144ab1bd06312edcde3f45 at {username='oryx', principal='oryx/127.0.236.125@KRBTEST.COM'} at 127.0.0.1:42420: session id = e7d2710f37144ab1bd06312edcde3f45-00000000000000000000000000000000
I20260806 15:03:29.989777 9974 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 15:03:29.990360 10594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 15:03:29.991498 10594 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45: tablet copy: Starting download of 0 data blocks...
I20260806 15:03:29.991634 10594 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45: tablet copy: Starting download of 1 WAL segments...
I20260806 15:03:29.992491 10594 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 15:03:29.993302 9974 tablet_copy_service.cc:342] P c92fd02274d84e3db9c69443339af692: Request end of tablet copy session e7d2710f37144ab1bd06312edcde3f45-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.0.236.125@KRBTEST.COM'} at 127.0.0.1:42420
I20260806 15:03:29.993359 9974 tablet_copy_service.cc:433] P c92fd02274d84e3db9c69443339af692: ending tablet copy session e7d2710f37144ab1bd06312edcde3f45-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e7d2710f37144ab1bd06312edcde3f45
I20260806 15:03:29.995576 10594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:29.996475 10731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:29.996511 10729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:29.998562 10733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:29.998648 10594 server_base.cc:1034] running on GCE node
I20260806 15:03:29.998798 10594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:29.998917 10594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:30.000029 10594 hybrid_clock.cc:648] HybridClock initialized: now 1786028610000026 us; error 25 us; skew 500 ppm
Aug 06 15:03:29 dist-test-slave-tc2s krb5kdc[9876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028609, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:30.001291 10594 init.cc:377] Logged in from keytab as oryx/127.0.236.125@KRBTEST.COM (short username oryx)
I20260806 15:03:30.001525 10594 webserver.cc:460] Webserver started at http://127.0.236.125:40559/ using document root <none> and password file <none>
I20260806 15:03:30.001631 10594 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:30.001662 10594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:30.002285 10594 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:03:30.002758 10746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:30.002872 10594 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:30.002930 10594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-1/wal
uuid: "e7d2710f37144ab1bd06312edcde3f45"
format_stamp: "Formatted at 2026-08-06 15:03:29 on dist-test-slave-tc2s"
server_key: "fc1f004dc67141add0fc5782b8372808"
server_key_iv: "0511ea2668ede40d4cf5685f11940ad0"
server_key_version: "encryptionkey@0"
I20260806 15:03:30.002978 10594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:30.018074 10594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:30.019562 10594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:30.021824 10594 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.125:38713
I20260806 15:03:30.022147 10812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.125:38713 every 8 connection(s)
I20260806 15:03:30.022207 10594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-1/data/info.pb
I20260806 15:03:30.023068 10813 sys_catalog.cc:263] Verifying existing consensus state
I20260806 15:03:30.023602 10813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45: Bootstrap starting.
I20260806 15:03:30.026669 10813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 15:03:30.026906 10813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45: Bootstrap complete.
I20260806 15:03:30.027122 10813 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: NON_VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: true } }
I20260806 15:03:30.027210 10813 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e7d2710f37144ab1bd06312edcde3f45, State: Initialized, Role: LEARNER
I20260806 15:03:30.027256 10813 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: NON_VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: true } }
I20260806 15:03:30.027436 10813 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:03:30.027434 10818 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: NON_VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: true } } }
I20260806 15:03:30.027596 10818 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [sys.catalog]: This master's current role is: LEARNER
I20260806 15:03:30.028983 10835 catalog_manager.cc:1259] Loaded cluster ID: fef1eb67d42c483abc65fd8ff931c254
I20260806 15:03:30.029027 10835 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45: loading cluster ID for follower catalog manager: success
I20260806 15:03:30.029881 10835 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45: acquiring CA information for follower catalog manager: success
I20260806 15:03:30.030174 10835 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 15:03:30.037307 10836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.031652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42260 (local address 127.0.236.125:38713)
0806 15:03:30.031754 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.031755 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.031763 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:30.031782 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.031782 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.031791 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.031842 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:30.032427 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.033036 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.033727 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.033980 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.034030 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.035088 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:30.035089 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:30.035090 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:30.035096 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:30.036336 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.036635 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.036636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.036636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.036691 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.036849 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.036850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.036850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.036954 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:30.037038 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.037137 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.037211 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":54,"threads_started":1}
WARNING: no policy specified for oryx/127.0.236.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.236.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.236.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.124.keytab.
Entry for principal oryx/127.0.236.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.124.keytab.
I20260806 15:03:30.064385 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.124:39837
--webserver_interface=127.0.236.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.124.keytab
--principal=oryx/127.0.236.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:39033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:35729,127.0.236.125:38713,127.0.236.124:39837
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:03:30.094913 10404 heartbeater.cc:503] Master 127.0.236.126:35729 was elected leader, sending a full tablet report...
W20260806 15:03:30.144572 10840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:30.144752 10840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:30.144773 10840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:30.145872 10840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:03:30.145947 10840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:30.145977 10840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:30.145998 10840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:03:30.146020 10840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:03:30.147497 10840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:39033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:35729,127.0.236.125:38713,127.0.236.124:39837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.124.keytab
--principal=oryx/127.0.236.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.124:39837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.236.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:30.147709 10840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:30.147899 10840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:30.152278 10849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:30.152359 10847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:30.152328 10840 server_base.cc:1034] running on GCE node
W20260806 15:03:30.152334 10846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:30.152702 10840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:30.152983 10840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:30.154112 10840 hybrid_clock.cc:648] HybridClock initialized: now 1786028610154100 us; error 28 us; skew 500 ppm
Aug 06 15:03:30 dist-test-slave-tc2s krb5kdc[9876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028610, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:30.156307 10840 init.cc:377] Logged in from keytab as oryx/127.0.236.124@KRBTEST.COM (short username oryx)
I20260806 15:03:30.156720 10840 webserver.cc:460] Webserver started at http://127.0.236.124:45995/ using document root <none> and password file <none>
I20260806 15:03:30.156921 10840 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:30.156970 10840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:30.157083 10840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:30.158046 10840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-2/data/instance:
uuid: "44b909a3dfab4801a2496d6f37cb3d8c"
format_stamp: "Formatted at 2026-08-06 15:03:30 on dist-test-slave-tc2s"
server_key: "8e10d8d840f8f407390fdaf7d541fb27"
server_key_iv: "ba0e7081b2ebc2c9f53d85ed04ba5130"
server_key_version: "encryptionkey@0"
I20260806 15:03:30.158370 10840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-2/wal/instance:
uuid: "44b909a3dfab4801a2496d6f37cb3d8c"
format_stamp: "Formatted at 2026-08-06 15:03:30 on dist-test-slave-tc2s"
server_key: "8e10d8d840f8f407390fdaf7d541fb27"
server_key_iv: "ba0e7081b2ebc2c9f53d85ed04ba5130"
server_key_version: "encryptionkey@0"
I20260806 15:03:30.159523 10840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:03:30.160125 10856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:30.160256 10840 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:03:30.160341 10840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-2/wal
uuid: "44b909a3dfab4801a2496d6f37cb3d8c"
format_stamp: "Formatted at 2026-08-06 15:03:30 on dist-test-slave-tc2s"
server_key: "8e10d8d840f8f407390fdaf7d541fb27"
server_key_iv: "ba0e7081b2ebc2c9f53d85ed04ba5130"
server_key_version: "encryptionkey@0"
I20260806 15:03:30.160405 10840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:30.175673 10840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:30.177161 10840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:30.177274 10840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:30.181143 10840 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.124:39837
I20260806 15:03:30.181174 10936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.124:39837 every 8 connection(s)
I20260806 15:03:30.181466 10840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-2/data/info.pb
I20260806 15:03:30.182336 10938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:30.183622 10938 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 15:03:30 dist-test-slave-tc2s krb5kdc[9876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028610, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.124@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM
I20260806 15:03:30.188680 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 10840
I20260806 15:03:30.191766 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.184264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42424 (local address 127.0.236.126:35729)
0806 15:03:30.184332 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.184334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.184427 (+ 93us) server_negotiation.cc:407] Connection header received
0806 15:03:30.184933 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.184935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.184946 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.184986 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:30.185645 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.186167 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.187059 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.187237 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.187281 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.189232 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:30.189234 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:30.189234 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:30.189242 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:30.190526 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.190885 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.190887 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.190887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.190949 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.191160 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.191161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.191161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.191273 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:30.191330 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.191587 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.191674 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 15:03:30.191893 10940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.184160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.0.1:42424)
0806 15:03:30.184309 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:30.184352 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:30.184825 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:30.185049 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:30.185052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:30.185333 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:30.185579 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.185584 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.186236 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.186237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.186999 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.187000 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.187298 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.187298 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.187394 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.187770 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:30.187778 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:30.189094 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:30.190601 (+ 1507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.190608 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.190610 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.190817 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.191003 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.191003 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.191004 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.191105 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.191328 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:30.191335 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:30.191468 (+ 133us) client_negotiation.cc:769] Sending connection context
0806 15:03:30.191555 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 15:03:30.191699 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":74,"threads_started":1}
I20260806 15:03:30.192518 10938 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 15:03:30 dist-test-slave-tc2s krb5kdc[9876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.236.124@KRBTEST.COM
Aug 06 15:03:30 dist-test-slave-tc2s krb5kdc[9876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028610, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.124@KRBTEST.COM for oryx/127.0.236.125@KRBTEST.COM
I20260806 15:03:30.196934 10941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.189154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57858 (local address 127.0.236.124:39837)
0806 15:03:30.189270 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.189274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.189290 (+ 16us) server_negotiation.cc:407] Connection header received
0806 15:03:30.189306 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.189307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.189316 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.189369 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:30.190056 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.190724 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.191435 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.191710 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.191764 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.194355 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:30.194362 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:30.194363 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:30.194368 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:30.195836 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.196188 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.196191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.196192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.196250 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.196408 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.196409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.196409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.196579 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:30.196706 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.196763 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.196834 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":67,"threads_started":1}
I20260806 15:03:30.199014 10940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.192739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:38713 (local address 127.0.0.1:42274)
0806 15:03:30.192865 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:30.192875 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:30.192927 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:30.193046 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:30.193047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:30.193172 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:30.193221 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.193223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.194122 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.194123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.194789 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.194792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.195252 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.195253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.195308 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.195603 (+ 295us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:30.195608 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:30.196933 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:30.198324 (+ 1391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.198325 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.198326 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.198477 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.198616 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.198616 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.198616 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.198658 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.198819 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:30.198820 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:30.198850 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 15:03:30.198892 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 15:03:30.198958 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110}
I20260806 15:03:30.199029 10836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.192814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42274 (local address 127.0.236.125:38713)
0806 15:03:30.192871 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.192872 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.192913 (+ 41us) server_negotiation.cc:407] Connection header received
0806 15:03:30.192988 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.192989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.193000 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.193041 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:30.193285 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.194033 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.194871 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.195184 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.195254 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.197038 (+ 1784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:30.197039 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:30.197040 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:30.197047 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:30.198253 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.198529 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.198529 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.198530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.198574 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.198695 (+ 121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.198695 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.198696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.198781 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:30.198823 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.198909 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.198970 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 15:03:30.199357 10938 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.124" port: 39837 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 15:03:30 dist-test-slave-tc2s krb5kdc[9876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028610, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.124@KRBTEST.COM for oryx/127.0.236.124@KRBTEST.COM
I20260806 15:03:30.204524 10940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.199562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:39837 (local address 127.0.0.1:57860)
0806 15:03:30.199601 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:30.199610 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:30.199651 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:30.199745 (+ 94us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:30.199745 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:30.199842 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:30.199867 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.199868 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.200390 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.200390 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.200822 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.200823 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.201089 (+ 266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.201090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.201121 (+ 31us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.201436 (+ 315us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:30.201441 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:30.202465 (+ 1024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:30.203672 (+ 1207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.203672 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.203673 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.203839 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.204069 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.204069 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.204070 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.204127 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.204314 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:30.204314 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:30.204344 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 15:03:30.204386 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 15:03:30.204454 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260806 15:03:30.204586 10941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.199588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57860 (local address 127.0.236.124:39837)
0806 15:03:30.199616 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.199617 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.199640 (+ 23us) server_negotiation.cc:407] Connection header received
0806 15:03:30.199695 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.199695 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.199703 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.199743 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:30.199918 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.200342 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.200871 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.201046 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.201085 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.202541 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:30.202542 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:30.202542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:30.202547 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:30.203610 (+ 1063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.203884 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.203885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.203885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.204013 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.204165 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.204165 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.204165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.204273 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:30.204317 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.204404 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.204494 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":13}
I20260806 15:03:30.205695 10938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c: Bootstrap starting.
I20260806 15:03:30.206460 10938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:30.206676 10938 log.cc:824] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:30.207244 10938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c: No bootstrap required, opened a new log
I20260806 15:03:30.208537 10938 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 39837 } }
I20260806 15:03:30.208642 10938 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:30.208669 10938 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44b909a3dfab4801a2496d6f37cb3d8c, State: Initialized, Role: FOLLOWER
I20260806 15:03:30.208750 10938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 39837 } }
I20260806 15:03:30.208961 10938 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:03:30.208948 10943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 39837 } } }
I20260806 15:03:30.209031 10943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [sys.catalog]: This master's current role is: FOLLOWER
W20260806 15:03:30.210497 10956 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:03:30.210542 10956 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:03:30.212318 10840 master_runner.cc:428] Detected that this master 44b909a3dfab4801a2496d6f37cb3d8c is joining an existing cluster
I20260806 15:03:30.212361 10840 master_runner.cc:432] Initiating AddMaster RPC to add 127.0.236.124:39837
I20260806 15:03:30.219893 10836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.213747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42280 (local address 127.0.236.125:38713)
0806 15:03:30.213782 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.213783 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.214306 (+ 523us) server_negotiation.cc:407] Connection header received
0806 15:03:30.214327 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.214327 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.214336 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.214389 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:30.214708 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.215240 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.216256 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.216516 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.216566 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.217503 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:30.217505 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:30.217505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:30.217511 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:30.218966 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.219252 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.219253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.219253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.219305 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.219459 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.219460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.219460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.219561 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:30.219614 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.219740 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.219804 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:03:30.219910 10967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.213580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:38713 (local address 127.0.0.1:42280)
0806 15:03:30.214230 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:30.214238 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:30.214275 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:30.214432 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:30.214432 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:30.214590 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:30.214638 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.214640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.215320 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.215321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.216177 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.216180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.216582 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.216583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.216649 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.217031 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:30.217035 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:30.217429 (+ 394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:30.219039 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.219040 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.219040 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.219188 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.219358 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.219359 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.219359 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.219409 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.219613 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:30.219614 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:30.219655 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 15:03:30.219713 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 15:03:30.219807 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":81,"threads_started":1}
I20260806 15:03:30.223731 10941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.213778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57862 (local address 127.0.236.124:39837)
0806 15:03:30.213809 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.213810 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.218044 (+ 4234us) server_negotiation.cc:407] Connection header received
0806 15:03:30.218091 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.218091 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.218101 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.218157 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:30.219002 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.219691 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.220628 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.220836 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.220885 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.221692 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:30.221692 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:30.221693 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:30.221697 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:30.222855 (+ 1158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.223177 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.223178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.223178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.223217 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.223372 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.223372 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.223372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.223459 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:30.223499 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.223623 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.223680 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:03:30.223850 10966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.213860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:39837 (local address 127.0.0.1:57862)
0806 15:03:30.217971 (+ 4111us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:30.217982 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:30.218045 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:30.218393 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:30.218394 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:30.218575 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:30.218907 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.218909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.219776 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.219777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.220563 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.220565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.220896 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.220896 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.220956 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.221267 (+ 311us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:30.221271 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:30.221623 (+ 352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:30.222918 (+ 1295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.222919 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.222919 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.223118 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.223265 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.223266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.223266 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.223326 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.223504 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:30.223504 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:30.223543 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 15:03:30.223609 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 15:03:30.223743 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4094,"mutex_wait_us":44,"thread_start_us":77,"threads_started":1}
I20260806 15:03:30.236356 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.213582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42434 (local address 127.0.236.126:35729)
0806 15:03:30.213625 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.213626 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.213733 (+ 107us) server_negotiation.cc:407] Connection header received
0806 15:03:30.213839 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.213841 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.213852 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.213943 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:30.214215 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.214956 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.215748 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.215932 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.215989 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.234242 (+ 18253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:30.234244 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:30.234244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:30.234252 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:30.235395 (+ 1143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.235693 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.235694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.235695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.235744 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.235902 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.235902 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.235903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.236011 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:30.236067 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.236197 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.236265 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 15:03:30.236374 10965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.213550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.0.1:42434)
0806 15:03:30.213679 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:30.213688 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:30.213756 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:30.213962 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:30.213963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:30.214095 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:30.214150 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.214151 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.215031 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.215032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.215674 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.215676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.216008 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.216009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.216067 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.216635 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:30.216638 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:30.234168 (+ 17530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:30.235475 (+ 1307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.235476 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.235476 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.235632 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.235796 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.235797 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.235797 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.235852 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.236065 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:30.236066 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:30.236108 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 15:03:30.236168 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 15:03:30.236265 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
Aug 06 15:03:30 dist-test-slave-tc2s krb5kdc[9876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028607, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.126@KRBTEST.COM for oryx/127.0.236.124@KRBTEST.COM
I20260806 15:03:30.244328 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.238121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:39837 (local address 127.0.0.1:57866)
0806 15:03:30.238169 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:30.238180 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:30.238218 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:30.238331 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:30.238332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:30.238500 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:30.238544 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.238545 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.239161 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.239162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.239976 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.239979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.240264 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.240265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.240338 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.240658 (+ 320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:30.240663 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:30.241839 (+ 1176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:30.243386 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.243387 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.243388 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.243540 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.243703 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.243704 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.243704 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.243745 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.243969 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:30.243969 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:30.244099 (+ 130us) client_negotiation.cc:769] Sending connection context
0806 15:03:30.244181 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 15:03:30.244260 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 15:03:30.244418 10941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.238173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57866 (local address 127.0.236.124:39837)
0806 15:03:30.238206 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.238208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.238216 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:30.238272 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.238273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.238280 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.238319 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:30.238601 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.239090 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.240042 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.240209 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.240253 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.241981 (+ 1728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:30.241983 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:30.241983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:30.241989 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:30.243315 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.243597 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.243597 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.243598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.243650 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.243789 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.243790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.243790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.243902 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:30.243966 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.244251 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.244331 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 15:03:30.244845 9945 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.0.236.124:39837: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c"
member_type: NON_VOTER
last_known_addr {
host: "127.0.236.124"
port: 39837
}
attrs {
promote: true
}
}
dest_uuid: "c92fd02274d84e3db9c69443339af692"
cas_config_opid_index: 5
I20260806 15:03:30.245070 9945 consensus_queue.cc:237] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: NON_VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: true } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: NON_VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: true } }
I20260806 15:03:30.245411 10714 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 44b909a3dfab4801a2496d6f37cb3d8c (127.0.236.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: NON_VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: true } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: NON_VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: true } } }
I20260806 15:03:30.245617 10898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:30.246263 10714 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.0.236.124:39837
I20260806 15:03:30.246337 10712 sys_catalog.cc:455] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c92fd02274d84e3db9c69443339af692" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: NON_VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: true } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: NON_VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: true } } }
I20260806 15:03:30.246414 10712 sys_catalog.cc:458] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:30.246501 10898 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [term 1 FOLLOWER]: Refusing update from remote peer c92fd02274d84e3db9c69443339af692: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 15:03:30.246661 10840 master.cc:537] Master@127.0.236.124:39837 shutting down...
I20260806 15:03:30.247212 10712 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: NON_VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 15:03:30.247920 9923 consensus_peers.cc:487] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 -> Peer 44b909a3dfab4801a2496d6f37cb3d8c (127.0.236.124:39837): Couldn't send request to peer 44b909a3dfab4801a2496d6f37cb3d8c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 15:03:30.249274 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.245438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:38713 (local address 127.0.0.1:42288)
0806 15:03:30.245518 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:30.245526 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:30.245562 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:30.245714 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:30.245715 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:30.245716 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:30.245745 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.245746 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.246572 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.246573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.247797 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.247800 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.249026 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.249028 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.249096 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.249121 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 15:03:30.249173 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 15:03:30.249208 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 15:03:30.249498 10836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.245528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42288 (local address 127.0.236.125:38713)
0806 15:03:30.245568 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.245569 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.245578 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:03:30.245600 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.245601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.245623 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.245692 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:30.245808 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.246489 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.247861 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.248425 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.248924 (+ 499us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.249000 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.249402 (+ 402us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.249440 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 15:03:30.249702 10777 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 1 LEARNER]: Refusing update from remote peer c92fd02274d84e3db9c69443339af692: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 15:03:30.249912 10712 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: NON_VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 15:03:30.250483 10777 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 44b909a3dfab4801a2496d6f37cb3d8c (127.0.236.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: NON_VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: true } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: NON_VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: true } } }
I20260806 15:03:30.251583 10970 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c92fd02274d84e3db9c69443339af692" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: NON_VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: true } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: NON_VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: true } } }
I20260806 15:03:30.251727 10970 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [sys.catalog]: This master's current role is: LEARNER
I20260806 15:03:30.251809 10818 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c92fd02274d84e3db9c69443339af692. Latest consensus state: current_term: 1 leader_uuid: "c92fd02274d84e3db9c69443339af692" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: NON_VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: true } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: NON_VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: true } } }
I20260806 15:03:30.251915 10818 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [sys.catalog]: This master's current role is: LEARNER
I20260806 15:03:30.252107 10712 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692: attempting to promote NON_VOTER e7d2710f37144ab1bd06312edcde3f45 to VOTER
I20260806 15:03:30.252278 10712 consensus_queue.cc:237] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: NON_VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: true } }
I20260806 15:03:30.252604 10840 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:30.252691 10840 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:30.252737 10840 tablet_replica.cc:330] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c: stopping tablet replica
I20260806 15:03:30.252785 10777 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 1 LEARNER]: Refusing update from remote peer c92fd02274d84e3db9c69443339af692: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 15:03:30.252959 10714 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260806 15:03:30.253038 9923 consensus_peers.cc:487] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 -> Peer 44b909a3dfab4801a2496d6f37cb3d8c (127.0.236.124:39837): Couldn't send request to peer 44b909a3dfab4801a2496d6f37cb3d8c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 15:03:30.253552 10714 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e7d2710f37144ab1bd06312edcde3f45 (127.0.236.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: NON_VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: true } } }
W20260806 15:03:30.253870 10858 connection.cc:441] server connection from 127.0.0.1:57858 torn down before Call kudu.master.MasterService.ListTables from 127.0.0.1:57858 (request call id 34) could send its response
I20260806 15:03:30.254410 10714 sys_catalog.cc:455] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c92fd02274d84e3db9c69443339af692" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: NON_VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: true } } }
I20260806 15:03:30.254392 10712 sys_catalog.cc:455] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c92fd02274d84e3db9c69443339af692" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: NON_VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: true } } }
I20260806 15:03:30.254513 10714 sys_catalog.cc:458] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:30.254523 10712 sys_catalog.cc:458] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:30.254948 10777 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e7d2710f37144ab1bd06312edcde3f45 (127.0.236.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: NON_VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: true } } }
I20260806 15:03:30.255718 10818 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c92fd02274d84e3db9c69443339af692" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: NON_VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: true } } }
I20260806 15:03:30.255802 10818 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:03:30.265650 10840 master.cc:552] Master@127.0.236.124:39837 shutdown complete.
I20260806 15:03:30.265686 10840 master_runner.cc:305] Clearing existing system tablet
I20260806 15:03:30.265790 10840 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:30.265827 10840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:30.266366 10840 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:30.266765 10975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:30.266877 10840 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:30.266927 10840 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:30.266948 10840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-2/wal
uuid: "44b909a3dfab4801a2496d6f37cb3d8c"
format_stamp: "Formatted at 2026-08-06 15:03:30 on dist-test-slave-tc2s"
server_key: "8e10d8d840f8f407390fdaf7d541fb27"
server_key_iv: "ba0e7081b2ebc2c9f53d85ed04ba5130"
server_key_version: "encryptionkey@0"
I20260806 15:03:30.267545 10840 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:30.268594 10840 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 15:03:30.268656 10840 log.cc:1196] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 15:03:30.268805 10840 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c: Deleting consensus metadata
I20260806 15:03:30.268921 10840 master_runner.cc:315] Copying system tablet from 127.0.236.126:35729
I20260806 15:03:30.269455 10840 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.236.126:35729
I20260806 15:03:30.275141 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.269759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42450 (local address 127.0.236.126:35729)
0806 15:03:30.269800 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.269802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.269872 (+ 70us) server_negotiation.cc:407] Connection header received
0806 15:03:30.269991 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.269992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.270002 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.270061 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:30.270362 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.270808 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.271774 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.271947 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.271989 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.272911 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:30.272912 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:30.272913 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:30.272919 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:30.274196 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.274536 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.274537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.274537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.274582 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.274748 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.274748 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.274749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.274836 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:30.274879 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.275034 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.275086 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 15:03:30.275200 10982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.269687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.0.1:42450)
0806 15:03:30.269790 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:30.269801 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:30.269870 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:30.270070 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:30.270071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:30.270250 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:30.270298 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.270300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.270886 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.270886 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.271700 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.271703 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.272008 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.272009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.272069 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.272442 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:30.272447 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:30.272798 (+ 351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:30.274265 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.274266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.274267 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.274472 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.274634 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.274634 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.274635 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.274698 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.274886 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:30.274887 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:30.274944 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 15:03:30.275017 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 15:03:30.275113 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":61,"threads_started":1}
I20260806 15:03:30.275452 9974 tablet_copy_service.cc:140] P c92fd02274d84e3db9c69443339af692: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 44b909a3dfab4801a2496d6f37cb3d8c ({username='oryx', principal='oryx/127.0.236.124@KRBTEST.COM'} at 127.0.0.1:42450)
I20260806 15:03:30.275501 9974 tablet_copy_service.cc:161] P c92fd02274d84e3db9c69443339af692: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 44b909a3dfab4801a2496d6f37cb3d8c at {username='oryx', principal='oryx/127.0.236.124@KRBTEST.COM'} at 127.0.0.1:42450: session id = 44b909a3dfab4801a2496d6f37cb3d8c-00000000000000000000000000000000
I20260806 15:03:30.275867 9974 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 15:03:30.276268 10840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 15:03:30.277050 10840 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c: tablet copy: Starting download of 0 data blocks...
I20260806 15:03:30.277176 10840 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c: tablet copy: Starting download of 1 WAL segments...
I20260806 15:03:30.277951 10840 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 15:03:30.278656 9974 tablet_copy_service.cc:342] P c92fd02274d84e3db9c69443339af692: Request end of tablet copy session 44b909a3dfab4801a2496d6f37cb3d8c-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.0.236.124@KRBTEST.COM'} at 127.0.0.1:42450
I20260806 15:03:30.278700 9974 tablet_copy_service.cc:433] P c92fd02274d84e3db9c69443339af692: ending tablet copy session 44b909a3dfab4801a2496d6f37cb3d8c-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 44b909a3dfab4801a2496d6f37cb3d8c
I20260806 15:03:30.280941 10840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:30.281908 10983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:30.281932 10984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:30.286562 10986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:30.286702 10840 server_base.cc:1034] running on GCE node
I20260806 15:03:30.286804 10840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:30.286958 10840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:30.288097 10840 hybrid_clock.cc:648] HybridClock initialized: now 1786028610288087 us; error 23 us; skew 500 ppm
Aug 06 15:03:30 dist-test-slave-tc2s krb5kdc[9876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028610, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:30.289366 10840 init.cc:377] Logged in from keytab as oryx/127.0.236.124@KRBTEST.COM (short username oryx)
I20260806 15:03:30.289645 10840 webserver.cc:460] Webserver started at http://127.0.236.124:40441/ using document root <none> and password file <none>
I20260806 15:03:30.289757 10840 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:30.289796 10840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:30.290508 10840 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:30.291018 10995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:30.291144 10840 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:30.291189 10840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-2/wal
uuid: "44b909a3dfab4801a2496d6f37cb3d8c"
format_stamp: "Formatted at 2026-08-06 15:03:30 on dist-test-slave-tc2s"
server_key: "8e10d8d840f8f407390fdaf7d541fb27"
server_key_iv: "ba0e7081b2ebc2c9f53d85ed04ba5130"
server_key_version: "encryptionkey@0"
I20260806 15:03:30.291239 10840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:30.321650 10840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:30.323065 10840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:30.326936 10840 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.124:39837
I20260806 15:03:30.326980 11069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.124:39837 every 8 connection(s)
I20260806 15:03:30.327235 10840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/master-2/data/info.pb
I20260806 15:03:30.328218 11073 sys_catalog.cc:263] Verifying existing consensus state
I20260806 15:03:30.328771 11073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c: Bootstrap starting.
I20260806 15:03:30.331626 11073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 15:03:30.331828 11073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c: Bootstrap complete.
I20260806 15:03:30.332024 11073 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: NON_VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: true } }
I20260806 15:03:30.332105 11073 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 44b909a3dfab4801a2496d6f37cb3d8c, State: Initialized, Role: LEARNER
I20260806 15:03:30.332136 11073 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: NON_VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: true } }
I20260806 15:03:30.332298 11073 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:03:30.332304 11076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: NON_VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: true } } }
I20260806 15:03:30.332418 11076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [sys.catalog]: This master's current role is: LEARNER
I20260806 15:03:30.333796 11089 catalog_manager.cc:1259] Loaded cluster ID: fef1eb67d42c483abc65fd8ff931c254
I20260806 15:03:30.333871 11089 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c: loading cluster ID for follower catalog manager: success
I20260806 15:03:30.334964 11089 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c: acquiring CA information for follower catalog manager: success
I20260806 15:03:30.335227 11089 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 15:03:30.360368 11090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.354984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57892 (local address 127.0.236.124:39837)
0806 15:03:30.355089 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.355091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.355099 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:30.355120 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.355121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.355129 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.355187 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:30.355686 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.356292 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.356967 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.357153 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.357206 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.358289 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:30.358291 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:30.358291 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:30.358298 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:30.359384 (+ 1086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.359684 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.359685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.359686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.359738 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.359893 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.359894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.359895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.360002 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:30.360088 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.360183 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.360263 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":45,"threads_started":1}
I20260806 15:03:30.360808 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 10032
I20260806 15:03:30.378696 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:36187
--local_ip_for_outbound_sockets=127.0.236.65
--tserver_master_addrs=127.0.236.126:35729
--webserver_port=39667
--webserver_interface=127.0.236.65
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.65.keytab
--principal=oryx/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:39033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.236.126:35729,127.0.236.125:38713
--tserver_master_addrs=127.0.236.126:35729,127.0.236.125:38713,127.0.236.124:39837 with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:30.458549 11091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:30.458719 11091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:30.458740 11091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:30.459836 11091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:30.459877 11091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:30.459906 11091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:03:30.461361 11091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:39033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.65.keytab
--principal=oryx/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:36187
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=39667
--tserver_master_addrs=127.0.236.126:35729,127.0.236.125:38713,127.0.236.124:39837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:30.461563 11091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:30.461761 11091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:30.466323 11102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:30.466310 11108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:30.466470 11091 server_base.cc:1034] running on GCE node
W20260806 15:03:30.466290 11105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:30.466722 11091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:30.466950 11091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:30.468082 11091 hybrid_clock.cc:648] HybridClock initialized: now 1786028610468073 us; error 26 us; skew 500 ppm
Aug 06 15:03:30 dist-test-slave-tc2s krb5kdc[9876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028610, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:30.470777 11091 init.cc:377] Logged in from keytab as oryx/127.0.236.65@KRBTEST.COM (short username oryx)
I20260806 15:03:30.471146 11091 webserver.cc:460] Webserver started at http://127.0.236.65:39667/ using document root <none> and password file <none>
I20260806 15:03:30.471349 11091 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:30.471408 11091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:30.472590 11091 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:30.473146 11116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:30.473286 11091 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:30.473364 11091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "120512845c1a446db9c5b628af55f0db"
format_stamp: "Formatted at 2026-08-06 15:03:27 on dist-test-slave-tc2s"
server_key: "9f778e6dd57585b66a834892494d73d0"
server_key_iv: "f670c7fcbacd15cc15f8256e84325eb3"
server_key_version: "encryptionkey@0"
I20260806 15:03:30.473658 11091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:30.493449 11091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:30.495180 11091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:30.495307 11091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:30.495504 11091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:30.495779 11091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:30.495812 11091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:30.495841 11091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:30.495863 11091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:30.501179 11091 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:36187
I20260806 15:03:30.501250 11282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:36187 every 8 connection(s)
I20260806 15:03:30.501533 11091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:03:30.503322 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 11091
I20260806 15:03:30.503495 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 10215
Aug 06 15:03:30 dist-test-slave-tc2s krb5kdc[9876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028610, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.124@KRBTEST.COM
Aug 06 15:03:30 dist-test-slave-tc2s krb5kdc[9876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028610, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.125@KRBTEST.COM
I20260806 15:03:30.519366 11090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.502528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:42909 (local address 127.0.236.124:39837)
0806 15:03:30.502575 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.502577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.502586 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:03:30.503086 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.503087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.503103 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.503155 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:30.511149 (+ 7994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.511780 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.512974 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.513209 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.513246 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.515818 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:30.515819 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:30.515820 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:30.515829 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:30.518151 (+ 2322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.518547 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.518548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.518548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.518598 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.518797 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.518798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.518799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.518899 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:30.518938 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.519214 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.519290 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 15:03:30.519795 11292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.502203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:39837 (local address 127.0.236.65:42909)
0806 15:03:30.502376 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:30.502406 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:30.502999 (+ 593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:30.503219 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:30.503223 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:30.510812 (+ 7589us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:30.511048 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.511053 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.511874 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.511875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.512889 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.512891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.513321 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.513322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.513430 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.513865 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:30.513874 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:30.515681 (+ 1807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:30.518224 (+ 2543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.518231 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.518232 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.518476 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.518650 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.518651 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.518651 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.518747 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.518962 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:30.518966 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:30.519098 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 15:03:30.519482 (+ 384us) client_negotiation.cc:240] Negotiation successful
0806 15:03:30.519621 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":92,"threads_started":1}
I20260806 15:03:30.521685 11285 heartbeater.cc:347] Connected to a master server at 127.0.236.124:39837
I20260806 15:03:30.521834 11285 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:30.522152 11285 heartbeater.cc:511] Master 127.0.236.124:39837 requested a full tablet report, sending...
I20260806 15:03:30.522770 11022 ts_manager.cc:194] Registered new tserver with Master: 120512845c1a446db9c5b628af55f0db (127.0.236.65:36187)
I20260806 15:03:30.524580 10836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.502593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:38213 (local address 127.0.236.125:38713)
0806 15:03:30.502645 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.502647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.502846 (+ 199us) server_negotiation.cc:407] Connection header received
0806 15:03:30.503208 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.503209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.503221 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.503270 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:30.514294 (+ 11024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.515069 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.515869 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.516271 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.516323 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.519063 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:30.519064 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:30.519065 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:30.519074 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:30.521566 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.523615 (+ 2049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.523617 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.523617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.523689 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.523960 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.523961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.523962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.524132 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:30.524197 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.524432 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.524507 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 15:03:30.524617 11290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.502204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:38713 (local address 127.0.236.65:38213)
0806 15:03:30.502791 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:30.502799 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:30.503011 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:30.503300 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:30.503301 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:30.514083 (+ 10782us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:30.514168 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.514171 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.515174 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.515174 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.515791 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.515792 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.516416 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.516417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.516493 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.517031 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:30.517037 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:30.518932 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:30.523251 (+ 4319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.523252 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.523252 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.523506 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.523779 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.523780 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.523780 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.523855 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.524238 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:30.524239 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:30.524293 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 15:03:30.524373 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 15:03:30.524491 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":189,"thread_start_us":75,"threads_started":1}
I20260806 15:03:30.525036 11288 heartbeater.cc:347] Connected to a master server at 127.0.236.125:38713
I20260806 15:03:30.525084 11288 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:30.525135 11288 heartbeater.cc:511] Master 127.0.236.125:38713 requested a full tablet report, sending...
I20260806 15:03:30.525810 10763 ts_manager.cc:194] Registered new tserver with Master: 120512845c1a446db9c5b628af55f0db (127.0.236.65:36187)
Aug 06 15:03:30 dist-test-slave-tc2s krb5kdc[9876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028610, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM
I20260806 15:03:30.535928 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:34923
--local_ip_for_outbound_sockets=127.0.236.66
--tserver_master_addrs=127.0.236.126:35729
--webserver_port=40617
--webserver_interface=127.0.236.66
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.66.keytab
--principal=oryx/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:39033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.236.126:35729,127.0.236.125:38713
--tserver_master_addrs=127.0.236.126:35729,127.0.236.125:38713,127.0.236.124:39837 with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:03:30.536338 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.502442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:35603 (local address 127.0.236.126:35729)
0806 15:03:30.502515 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.502517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.502745 (+ 228us) server_negotiation.cc:407] Connection header received
0806 15:03:30.503325 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.503326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.503336 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.503400 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:30.530365 (+ 26965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.531002 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.531873 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.532184 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.532249 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.534136 (+ 1887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:30.534137 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:30.534138 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:30.534145 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:30.535246 (+ 1101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.535585 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.535586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.535586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.535635 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.535796 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.535796 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.535797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.535930 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:30.536000 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.536168 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.536249 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 15:03:30.536342 11294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.502203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.65:35603)
0806 15:03:30.502659 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:30.502668 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:30.503004 (+ 336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:30.530073 (+ 27069us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:30.530074 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:30.530224 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:30.530274 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.530275 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.531093 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.531095 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.531786 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.531788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.532247 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.532248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.532306 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.532692 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:30.532696 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:30.534057 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:30.535324 (+ 1267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.535325 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.535325 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.535521 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.535691 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.535691 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.535691 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.535745 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.535988 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:30.535989 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:30.536030 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 15:03:30.536128 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 15:03:30.536221 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":3,"thread_start_us":79,"threads_started":1}
I20260806 15:03:30.536715 11286 heartbeater.cc:347] Connected to a master server at 127.0.236.126:35729
I20260806 15:03:30.536768 11286 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:30.536829 11286 heartbeater.cc:511] Master 127.0.236.126:35729 requested a full tablet report, sending...
I20260806 15:03:30.537111 9945 ts_manager.cc:194] Re-registered known tserver with Master: 120512845c1a446db9c5b628af55f0db (127.0.236.65:36187)
I20260806 15:03:30.537536 9945 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.236.65@KRBTEST.COM'} at 127.0.236.65:35603
W20260806 15:03:30.615132 11295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:30.615315 11295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:30.615336 11295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:30.616393 11295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:30.616425 11295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:30.616456 11295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:03:30.617944 11295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:39033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.66.keytab
--principal=oryx/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:34923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=40617
--tserver_master_addrs=127.0.236.126:35729,127.0.236.125:38713,127.0.236.124:39837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:30.618119 11295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:30.618281 11295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:30.622756 11302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:30.622756 11305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:30.622807 11303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:30.622965 11295 server_base.cc:1034] running on GCE node
I20260806 15:03:30.623183 11295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:30.623387 11295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:30.624522 11295 hybrid_clock.cc:648] HybridClock initialized: now 1786028610624514 us; error 32 us; skew 500 ppm
Aug 06 15:03:30 dist-test-slave-tc2s krb5kdc[9876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028610, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:30.626801 11295 init.cc:377] Logged in from keytab as oryx/127.0.236.66@KRBTEST.COM (short username oryx)
I20260806 15:03:30.627265 11295 webserver.cc:460] Webserver started at http://127.0.236.66:40617/ using document root <none> and password file <none>
I20260806 15:03:30.627489 11295 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:30.627571 11295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:30.628896 11295 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:30.629427 11316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:30.629575 11295 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:30.629652 11295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "b7ccff12a0d44255a60b61e0e3287c5c"
format_stamp: "Formatted at 2026-08-06 15:03:29 on dist-test-slave-tc2s"
server_key: "2254db597bfcf0e41be383f40597efe3"
server_key_iv: "eab5433e09ad47f25695507b41106ac2"
server_key_version: "encryptionkey@0"
I20260806 15:03:30.629979 11295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:30.640228 11295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:30.641793 11295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:30.641942 11295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:30.642123 11295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:30.642400 11295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:30.642454 11295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:30.642479 11295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:30.642500 11295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:30.646118 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.642652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:39837 (local address 127.0.0.1:57894)
0806 15:03:30.642698 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:30.642710 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:30.642756 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:30.643060 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:30.643061 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:30.643062 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:30.643105 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.643107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.643991 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.643992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.645072 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.645075 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.645849 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.645851 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.645936 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.645941 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 15:03:30.645990 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 15:03:30.646042 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 15:03:30.646126 11090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.642838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57894 (local address 127.0.236.124:39837)
0806 15:03:30.642910 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.642912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.642923 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:03:30.642954 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.642955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.642983 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.643054 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:30.643150 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.643913 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.645165 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.645764 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.645842 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.645947 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.646008 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.646054 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 15:03:30.646425 11035 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 15:03:30.646595 11076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c92fd02274d84e3db9c69443339af692. Latest consensus state: current_term: 1 leader_uuid: "c92fd02274d84e3db9c69443339af692" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: NON_VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: true } } }
I20260806 15:03:30.646696 11076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [sys.catalog]: This master's current role is: LEARNER
I20260806 15:03:30.648422 11295 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:34923
I20260806 15:03:30.648485 11474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:34923 every 8 connection(s)
I20260806 15:03:30.648753 11295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
I20260806 15:03:30.650848 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 11295
I20260806 15:03:30.651017 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 10411
Aug 06 15:03:30 dist-test-slave-tc2s krb5kdc[9876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028610, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM
Aug 06 15:03:30 dist-test-slave-tc2s krb5kdc[9876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028610, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.125@KRBTEST.COM
Aug 06 15:03:30 dist-test-slave-tc2s krb5kdc[9876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028610, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.124@KRBTEST.COM
I20260806 15:03:30.661424 11090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.649393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:55413 (local address 127.0.236.124:39837)
0806 15:03:30.649692 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.649694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.650118 (+ 424us) server_negotiation.cc:407] Connection header received
0806 15:03:30.650867 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.650867 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.650877 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.650906 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:30.653695 (+ 2789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.654385 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.655766 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.656150 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.656227 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.658365 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:30.658366 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:30.658367 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:30.658375 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:30.659849 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.660315 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.660316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.660316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.660392 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.660717 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.660718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.660719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.660849 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:30.660889 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.661258 (+ 369us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.661336 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279}
I20260806 15:03:30.661464 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.649580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:53449 (local address 127.0.236.126:35729)
0806 15:03:30.649628 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.649630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.650072 (+ 442us) server_negotiation.cc:407] Connection header received
0806 15:03:30.650866 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.650867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.650877 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.650906 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:30.651742 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.652298 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.653713 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.654020 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.654063 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.656921 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:30.656922 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:30.656923 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:30.656930 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:30.659828 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.660722 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.660723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.660723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.660773 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.660962 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.660963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.660964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.661083 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:30.661125 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.661308 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.661373 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 15:03:30.662003 11482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.649346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.66:53449)
0806 15:03:30.649996 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:30.650004 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:30.650776 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:30.651070 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:30.651074 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:30.651422 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:30.651655 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.651660 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.652393 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.652394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.653369 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.653372 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.654144 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.654145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.654277 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.654720 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:30.654730 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:30.656789 (+ 2059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:30.660453 (+ 3664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.660454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.660454 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.660645 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.660840 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.660841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.660841 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.660894 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.661169 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:30.661170 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:30.661203 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 15:03:30.661577 (+ 374us) client_negotiation.cc:240] Negotiation successful
0806 15:03:30.661761 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":106,"threads_started":1}
I20260806 15:03:30.662003 11480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.649346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:39837 (local address 127.0.236.66:55413)
0806 15:03:30.650078 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:30.650084 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:30.650772 (+ 688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:30.651070 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:30.651074 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:30.653575 (+ 2501us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:30.653619 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.653621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.654935 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.654946 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.655669 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.655673 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.656251 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.656253 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.656321 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.656796 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:30.656800 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:30.658147 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:30.659930 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.659948 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.659950 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.660224 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.660460 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.660461 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.660461 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.660614 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.660933 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:30.660951 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:30.661111 (+ 160us) client_negotiation.cc:769] Sending connection context
0806 15:03:30.661677 (+ 566us) client_negotiation.cc:240] Negotiation successful
0806 15:03:30.661807 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"mutex_wait_us":51,"spinlock_wait_cycles":5888,"thread_start_us":112,"threads_started":1}
I20260806 15:03:30.662551 11477 heartbeater.cc:347] Connected to a master server at 127.0.236.126:35729
I20260806 15:03:30.662663 11477 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:30.662796 11475 heartbeater.cc:347] Connected to a master server at 127.0.236.124:39837
I20260806 15:03:30.662842 11475 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:30.662894 11475 heartbeater.cc:511] Master 127.0.236.124:39837 requested a full tablet report, sending...
I20260806 15:03:30.662889 11477 heartbeater.cc:511] Master 127.0.236.126:35729 requested a full tablet report, sending...
I20260806 15:03:30.663256 9945 ts_manager.cc:194] Re-registered known tserver with Master: b7ccff12a0d44255a60b61e0e3287c5c (127.0.236.66:34923)
I20260806 15:03:30.663306 11022 ts_manager.cc:194] Registered new tserver with Master: b7ccff12a0d44255a60b61e0e3287c5c (127.0.236.66:34923)
I20260806 15:03:30.663672 9945 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.236.66@KRBTEST.COM'} at 127.0.236.66:53449
I20260806 15:03:30.668809 11481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.649345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:38713 (local address 127.0.236.66:39521)
0806 15:03:30.649526 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:30.649563 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:30.650761 (+ 1198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:30.651072 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:30.651074 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:30.651611 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:30.651693 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.651696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.652854 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.652855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.653851 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.653854 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.654477 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.654478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.654549 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.654959 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:30.654965 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:30.657366 (+ 2401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:30.666319 (+ 8953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.666321 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.666321 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.666518 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.666854 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.666854 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.666855 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.666929 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.668511 (+ 1582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:30.668512 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:30.668556 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 15:03:30.668625 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 15:03:30.668722 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":111,"threads_started":1}
I20260806 15:03:30.668870 10836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.649643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:39521 (local address 127.0.236.125:38713)
0806 15:03:30.649691 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.649693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.649702 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:03:30.650882 (+ 1180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.650884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.650896 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.650938 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:30.651802 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.652743 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.653999 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.654345 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.654391 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.657488 (+ 3097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:30.657489 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:30.657490 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:30.657498 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:30.660341 (+ 2843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.666635 (+ 6294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.666637 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.666637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.666743 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.667028 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.667029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.667029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.667245 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:30.667356 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.668702 (+ 1346us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.668783 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 15:03:30.669299 11478 heartbeater.cc:347] Connected to a master server at 127.0.236.125:38713
I20260806 15:03:30.669350 11478 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:30.669410 11478 heartbeater.cc:511] Master 127.0.236.125:38713 requested a full tablet report, sending...
I20260806 15:03:30.669656 10763 ts_manager.cc:194] Registered new tserver with Master: b7ccff12a0d44255a60b61e0e3287c5c (127.0.236.66:34923)
I20260806 15:03:30.672935 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:35007
--local_ip_for_outbound_sockets=127.0.236.67
--tserver_master_addrs=127.0.236.126:35729
--webserver_port=38517
--webserver_interface=127.0.236.67
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.67.keytab
--principal=oryx/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:39033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.236.126:35729,127.0.236.125:38713
--tserver_master_addrs=127.0.236.126:35729,127.0.236.125:38713,127.0.236.124:39837 with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:30.751618 11483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:30.751801 11483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:30.751821 11483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:30.752945 11483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:30.752986 11483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:30.753010 11483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:03:30.754424 11483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:39033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/krb5kdc/oryx_127.0.236.67.keytab
--principal=oryx/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:35007
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=38517
--tserver_master_addrs=127.0.236.126:35729,127.0.236.125:38713,127.0.236.124:39837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:30.754607 11483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:30.754830 11483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:30.759431 11498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:30.759441 11496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:30.759447 11494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:30.759588 11483 server_base.cc:1034] running on GCE node
I20260806 15:03:30.759845 11483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:30.760037 11483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:30.761170 11483 hybrid_clock.cc:648] HybridClock initialized: now 1786028610761156 us; error 28 us; skew 500 ppm
Aug 06 15:03:30 dist-test-slave-tc2s krb5kdc[9876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028610, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:30.763406 11483 init.cc:377] Logged in from keytab as oryx/127.0.236.67@KRBTEST.COM (short username oryx)
I20260806 15:03:30.763775 11483 webserver.cc:460] Webserver started at http://127.0.236.67:38517/ using document root <none> and password file <none>
I20260806 15:03:30.763960 11483 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:30.764014 11483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:30.765237 11483 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:30.765719 11508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:30.765868 11483 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:30.765983 11483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "8194526e9ac244b7b939dadfa140b2c7"
format_stamp: "Formatted at 2026-08-06 15:03:29 on dist-test-slave-tc2s"
server_key: "7c0c6c6b4918d6a040582d88f0cda804"
server_key_iv: "dcdb113b584426a143628d4f2ec99c4f"
server_key_version: "encryptionkey@0"
I20260806 15:03:30.766266 11483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:30.789028 11483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:30.790633 11483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:30.790756 11483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:30.790967 11483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:30.791249 11483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:30.791280 11483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:30.791307 11483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:30.791323 11483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:30.796506 11483 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:35007
I20260806 15:03:30.796547 11670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:35007 every 8 connection(s)
I20260806 15:03:30.796819 11483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
I20260806 15:03:30.797727 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 11483
Aug 06 15:03:30 dist-test-slave-tc2s krb5kdc[9876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028610, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.125@KRBTEST.COM
Aug 06 15:03:30 dist-test-slave-tc2s krb5kdc[9876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028610, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.124@KRBTEST.COM
Aug 06 15:03:30 dist-test-slave-tc2s krb5kdc[9876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028610, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM
I20260806 15:03:30.808885 11679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.797431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:38713 (local address 127.0.236.67:60235)
0806 15:03:30.797925 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:30.797936 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:30.798411 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:30.798689 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:30.798690 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:30.799143 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:30.799539 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.799542 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.800480 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.800481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.801143 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.801145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.801575 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.801577 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.801704 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.802182 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:30.802192 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:30.804758 (+ 2566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:30.807294 (+ 2536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.807301 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.807303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.807605 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.807817 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.807817 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.807818 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.807944 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.808235 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:30.808241 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:30.808401 (+ 160us) client_negotiation.cc:769] Sending connection context
0806 15:03:30.808514 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 15:03:30.808668 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":253,"thread_start_us":258,"threads_started":1}
I20260806 15:03:30.809526 10836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.797563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:60235 (local address 127.0.236.125:38713)
0806 15:03:30.797672 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.797674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.797992 (+ 318us) server_negotiation.cc:407] Connection header received
0806 15:03:30.798582 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.798584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.798596 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.798642 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:30.799672 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.800397 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.801225 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.801510 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.801566 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.804864 (+ 3298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:30.804865 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:30.804866 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:30.804872 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:30.806961 (+ 2089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.807688 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.807690 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.807690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.807750 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.808017 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.808018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.808019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.808149 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:30.808206 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.809323 (+ 1117us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.809400 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 15:03:30.810016 11674 heartbeater.cc:347] Connected to a master server at 127.0.236.125:38713
I20260806 15:03:30.810097 11674 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:30.810103 11090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.797589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:34991 (local address 127.0.236.124:39837)
0806 15:03:30.797965 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.797967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.797977 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:03:30.798508 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.798509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.798521 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.798551 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:30.799494 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.800066 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.801306 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.801572 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.801613 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.806796 (+ 5183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:30.806797 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:30.806797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:30.806804 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:30.808755 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.809286 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.809287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.809287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.809345 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.809539 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.809539 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.809540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.809669 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:30.809706 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.809928 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.810021 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346}
I20260806 15:03:30.810276 11674 heartbeater.cc:511] Master 127.0.236.125:38713 requested a full tablet report, sending...
I20260806 15:03:30.810437 11685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.798890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42296 (local address 127.0.236.125:38713)
0806 15:03:30.799023 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.799025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.801315 (+ 2290us) server_negotiation.cc:407] Connection header received
0806 15:03:30.801417 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.801418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.801431 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.801487 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:30.802292 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.803049 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.804327 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.804675 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.804749 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.807365 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:30.807367 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:30.807367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:30.807374 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:30.809076 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.809451 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.809452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.809452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.809527 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.809718 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.809719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.809719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.809836 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:30.809960 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.810262 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.810339 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
I20260806 15:03:30.810497 11678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.797431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:39837 (local address 127.0.236.67:34991)
0806 15:03:30.797798 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:30.797837 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:30.798407 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:30.798683 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:30.798688 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:30.799115 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:30.799375 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.799381 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.800164 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.800164 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.801209 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.801211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.801688 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.801688 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.801766 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.802483 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:30.802488 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:30.806436 (+ 3948us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:30.809006 (+ 2570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.809007 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.809007 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.809209 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.809417 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.809417 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.809417 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.809472 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.809750 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:30.809751 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:30.809793 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 15:03:30.810279 (+ 486us) client_negotiation.cc:240] Negotiation successful
0806 15:03:30.810370 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":2,"thread_start_us":315,"threads_started":1}
I20260806 15:03:30.810817 10762 ts_manager.cc:194] Registered new tserver with Master: 8194526e9ac244b7b939dadfa140b2c7 (127.0.236.67:35007)
I20260806 15:03:30.810897 11671 heartbeater.cc:347] Connected to a master server at 127.0.236.124:39837
I20260806 15:03:30.810947 11671 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:30.811015 11671 heartbeater.cc:511] Master 127.0.236.124:39837 requested a full tablet report, sending...
I20260806 15:03:30.811264 11022 ts_manager.cc:194] Registered new tserver with Master: 8194526e9ac244b7b939dadfa140b2c7 (127.0.236.67:35007)
I20260806 15:03:30.812218 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.797464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:42495 (local address 127.0.236.126:35729)
0806 15:03:30.797629 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.797632 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.798367 (+ 735us) server_negotiation.cc:407] Connection header received
0806 15:03:30.798508 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.798509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.798521 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.798551 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:30.799600 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.800373 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.801624 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.802327 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.802391 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.807051 (+ 4660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:30.807053 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:30.807053 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:30.807059 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:30.809703 (+ 2644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.811483 (+ 1780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.811485 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.811485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.811541 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.811717 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.811718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.811718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.811846 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:30.811894 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.812063 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.812134 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260806 15:03:30.812351 11677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.797431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.67:42495)
0806 15:03:30.798302 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:30.798311 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:30.798406 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:30.798684 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:30.798688 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:30.799166 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:30.799521 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.799523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.800455 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.800457 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.801303 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.801305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.803196 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.803197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.803258 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.805066 (+ 1808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:30.805071 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:30.806601 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:30.809998 (+ 3397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.809998 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.809999 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.811395 (+ 1396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.811601 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:30.811602 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:30.811603 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:30.811666 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:30.811910 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:30.811911 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:30.811961 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 15:03:30.812017 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 15:03:30.812106 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"mutex_wait_us":27,"spinlock_wait_cycles":6912,"thread_start_us":138,"threads_started":1}
I20260806 15:03:30.812659 11672 heartbeater.cc:347] Connected to a master server at 127.0.236.126:35729
I20260806 15:03:30.812708 11672 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:30.812765 11672 heartbeater.cc:511] Master 127.0.236.126:35729 requested a full tablet report, sending...
I20260806 15:03:30.812911 11691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.799665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57908 (local address 127.0.236.124:39837)
0806 15:03:30.800503 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.800505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.801528 (+ 1023us) server_negotiation.cc:407] Connection header received
0806 15:03:30.801930 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.801931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.801952 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.802024 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:30.803318 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.804037 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.805368 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.805661 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.805718 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.807118 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:30.807119 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:30.807120 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:30.807127 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:30.811662 (+ 4535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.811973 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.811974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.811974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.812034 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.812239 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.812240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.812240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.812358 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:30.812455 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.812755 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.812827 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":819,"thread_start_us":57,"threads_started":1}
I20260806 15:03:30.812974 9945 ts_manager.cc:194] Re-registered known tserver with Master: 8194526e9ac244b7b939dadfa140b2c7 (127.0.236.67:35007)
I20260806 15:03:30.813333 9945 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.236.67@KRBTEST.COM'} at 127.0.236.67:42495
I20260806 15:03:30.814105 11690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.799418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42458 (local address 127.0.236.126:35729)
0806 15:03:30.802830 (+ 3412us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.802832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.802982 (+ 150us) server_negotiation.cc:407] Connection header received
0806 15:03:30.803002 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.803002 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.803014 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.803067 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:30.803792 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.804547 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.805555 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.805840 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.805916 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.807993 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:30.807995 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:30.807995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:30.808002 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:30.812949 (+ 4947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.813406 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.813407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.813407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.813470 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.813667 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.813667 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.813667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.813761 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:30.813830 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.813982 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.814045 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3390,"thread_start_us":69,"threads_started":1}
I20260806 15:03:30.815495 9945 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42458:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:03:30.816934 9945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:03:30.817585 10971 sys_catalog.cc:455] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c92fd02274d84e3db9c69443339af692" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: NON_VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: true } } }
I20260806 15:03:30.817663 10971 sys_catalog.cc:458] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:30.818964 11694 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692: attempting to promote NON_VOTER 44b909a3dfab4801a2496d6f37cb3d8c to VOTER
I20260806 15:03:30.819099 11694 consensus_queue.cc:237] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: false } }
I20260806 15:03:30.819993 11035 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [term 1 LEARNER]: Refusing update from remote peer c92fd02274d84e3db9c69443339af692: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 15:03:30.820089 10777 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 1 FOLLOWER]: Refusing update from remote peer c92fd02274d84e3db9c69443339af692: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 15:03:30.820255 10971 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 15:03:30.820371 11693 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 15:03:30.821041 11694 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 44b909a3dfab4801a2496d6f37cb3d8c (127.0.236.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: false } } }
I20260806 15:03:30.821202 10777 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 44b909a3dfab4801a2496d6f37cb3d8c (127.0.236.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: false } } }
I20260806 15:03:30.822022 11694 sys_catalog.cc:455] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c92fd02274d84e3db9c69443339af692" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: false } } }
I20260806 15:03:30.822105 11694 sys_catalog.cc:458] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:30.822162 11694 sys_catalog.cc:455] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c92fd02274d84e3db9c69443339af692" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: false } } }
I20260806 15:03:30.822207 11694 sys_catalog.cc:458] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:30.822095 11705 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c92fd02274d84e3db9c69443339af692" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: false } } }
I20260806 15:03:30.822245 11705 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:03:30.822253 11694 sys_catalog.cc:455] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c92fd02274d84e3db9c69443339af692" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: false } } }
I20260806 15:03:30.822296 11694 sys_catalog.cc:458] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:30.822640 11035 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 44b909a3dfab4801a2496d6f37cb3d8c (127.0.236.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: false } } }
I20260806 15:03:30.823694 11076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c92fd02274d84e3db9c69443339af692" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: false } } }
I20260806 15:03:30.823767 11076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:03:30.827193 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.822802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:36187 (local address 127.0.0.1:60350)
0806 15:03:30.822933 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:30.822954 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:30.822992 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:30.823456 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:30.823457 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:30.823459 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:30.823501 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.823502 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.825046 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.825047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.826155 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.826157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.826975 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.826976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.827035 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.827039 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:03:30.827091 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 15:03:30.827129 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":3}
I20260806 15:03:30.827214 11712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.823137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60350 (local address 127.0.236.65:36187)
0806 15:03:30.823314 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.823318 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.823327 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:03:30.823360 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.823362 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.823374 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.823446 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:30.823553 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.824974 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.826245 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.826904 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.826968 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.827040 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.827106 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.827140 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":92,"threads_started":1}
I20260806 15:03:30.827410 11708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.822858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:35007 (local address 127.0.0.1:52666)
0806 15:03:30.822977 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:30.822986 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:30.823031 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:30.824026 (+ 995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:30.824027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:30.824029 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:30.824065 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.824067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.825269 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.825270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.826351 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.826353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.827163 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.827164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.827224 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.827228 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:03:30.827276 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 15:03:30.827320 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":69,"threads_started":1}
I20260806 15:03:30.827771 11713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.823387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52666 (local address 127.0.236.67:35007)
0806 15:03:30.823873 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.823878 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.823888 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:03:30.823921 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.823923 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.823947 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.824011 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:30.824146 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.825193 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.826424 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.827093 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.827150 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.827218 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.827634 (+ 416us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.827674 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":73,"threads_started":1}
I20260806 15:03:30.828243 11183 tablet_service.cc:1467] Processing CreateTablet for tablet 164d277c91b8405bb48a91a5fe2c9d9f (DEFAULT_TABLE table=test-table [id=c35bdea5150944deb814181e8ed850c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:30.828403 11709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.823031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:34923 (local address 127.0.0.1:49986)
0806 15:03:30.823134 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:30.823142 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:30.823177 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:30.824244 (+ 1067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:30.824245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:30.824246 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:30.824285 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.824286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.825994 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.825995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.826889 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.826892 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.828155 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.828157 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.828228 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.828233 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 15:03:30.828299 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 15:03:30.828336 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":75,"threads_started":1}
I20260806 15:03:30.828430 11716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.823541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49986 (local address 127.0.236.66:34923)
0806 15:03:30.824102 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.824106 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.824114 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:30.824151 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.824153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.824167 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.824242 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:30.824337 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.825355 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.827354 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.828069 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.828143 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.828211 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.828314 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.828355 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":81,"threads_started":1}
I20260806 15:03:30.828387 11573 tablet_service.cc:1467] Processing CreateTablet for tablet 164d277c91b8405bb48a91a5fe2c9d9f (DEFAULT_TABLE table=test-table [id=c35bdea5150944deb814181e8ed850c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:30.828619 11183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 164d277c91b8405bb48a91a5fe2c9d9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:30.828753 11573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 164d277c91b8405bb48a91a5fe2c9d9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:30.829039 11383 tablet_service.cc:1467] Processing CreateTablet for tablet 164d277c91b8405bb48a91a5fe2c9d9f (DEFAULT_TABLE table=test-table [id=c35bdea5150944deb814181e8ed850c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:30.829365 11383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 164d277c91b8405bb48a91a5fe2c9d9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:30.832100 11718 tablet_bootstrap.cc:492] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db: Bootstrap starting.
I20260806 15:03:30.832816 11717 tablet_bootstrap.cc:492] T 164d277c91b8405bb48a91a5fe2c9d9f P 8194526e9ac244b7b939dadfa140b2c7: Bootstrap starting.
I20260806 15:03:30.832954 11720 tablet_bootstrap.cc:492] T 164d277c91b8405bb48a91a5fe2c9d9f P b7ccff12a0d44255a60b61e0e3287c5c: Bootstrap starting.
I20260806 15:03:30.833025 11718 tablet_bootstrap.cc:654] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:30.833336 11718 log.cc:824] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:30.833725 11720 tablet_bootstrap.cc:654] T 164d277c91b8405bb48a91a5fe2c9d9f P b7ccff12a0d44255a60b61e0e3287c5c: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:30.833787 11717 tablet_bootstrap.cc:654] T 164d277c91b8405bb48a91a5fe2c9d9f P 8194526e9ac244b7b939dadfa140b2c7: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:30.834105 11720 log.cc:824] T 164d277c91b8405bb48a91a5fe2c9d9f P b7ccff12a0d44255a60b61e0e3287c5c: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:30.834117 11717 log.cc:824] T 164d277c91b8405bb48a91a5fe2c9d9f P 8194526e9ac244b7b939dadfa140b2c7: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:30.834450 11718 tablet_bootstrap.cc:492] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db: No bootstrap required, opened a new log
I20260806 15:03:30.834525 11718 ts_tablet_manager.cc:1397] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:03:30.834839 11717 tablet_bootstrap.cc:492] T 164d277c91b8405bb48a91a5fe2c9d9f P 8194526e9ac244b7b939dadfa140b2c7: No bootstrap required, opened a new log
I20260806 15:03:30.834957 11717 ts_tablet_manager.cc:1397] T 164d277c91b8405bb48a91a5fe2c9d9f P 8194526e9ac244b7b939dadfa140b2c7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:03:30.835089 11720 tablet_bootstrap.cc:492] T 164d277c91b8405bb48a91a5fe2c9d9f P b7ccff12a0d44255a60b61e0e3287c5c: No bootstrap required, opened a new log
I20260806 15:03:30.835150 11720 ts_tablet_manager.cc:1397] T 164d277c91b8405bb48a91a5fe2c9d9f P b7ccff12a0d44255a60b61e0e3287c5c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 15:03:30.836462 11718 raft_consensus.cc:348] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "120512845c1a446db9c5b628af55f0db" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36187 } } peers { permanent_uuid: "8194526e9ac244b7b939dadfa140b2c7" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 35007 } } peers { permanent_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34923 } }
I20260806 15:03:30.836582 11718 raft_consensus.cc:374] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:30.836614 11718 raft_consensus.cc:729] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 120512845c1a446db9c5b628af55f0db, State: Initialized, Role: FOLLOWER
I20260806 15:03:30.836745 11718 consensus_queue.cc:260] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "120512845c1a446db9c5b628af55f0db" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36187 } } peers { permanent_uuid: "8194526e9ac244b7b939dadfa140b2c7" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 35007 } } peers { permanent_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34923 } }
I20260806 15:03:30.836997 11718 ts_tablet_manager.cc:1428] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 15:03:30.837165 11286 heartbeater.cc:503] Master 127.0.236.126:35729 was elected leader, sending a full tablet report...
I20260806 15:03:30.837415 11720 raft_consensus.cc:348] T 164d277c91b8405bb48a91a5fe2c9d9f P b7ccff12a0d44255a60b61e0e3287c5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "120512845c1a446db9c5b628af55f0db" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36187 } } peers { permanent_uuid: "8194526e9ac244b7b939dadfa140b2c7" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 35007 } } peers { permanent_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34923 } }
I20260806 15:03:30.837651 11720 raft_consensus.cc:374] T 164d277c91b8405bb48a91a5fe2c9d9f P b7ccff12a0d44255a60b61e0e3287c5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:30.837746 11720 raft_consensus.cc:729] T 164d277c91b8405bb48a91a5fe2c9d9f P b7ccff12a0d44255a60b61e0e3287c5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7ccff12a0d44255a60b61e0e3287c5c, State: Initialized, Role: FOLLOWER
I20260806 15:03:30.837913 11720 consensus_queue.cc:260] T 164d277c91b8405bb48a91a5fe2c9d9f P b7ccff12a0d44255a60b61e0e3287c5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "120512845c1a446db9c5b628af55f0db" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36187 } } peers { permanent_uuid: "8194526e9ac244b7b939dadfa140b2c7" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 35007 } } peers { permanent_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34923 } }
I20260806 15:03:30.838225 11720 ts_tablet_manager.cc:1428] T 164d277c91b8405bb48a91a5fe2c9d9f P b7ccff12a0d44255a60b61e0e3287c5c: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 15:03:30.838568 11477 heartbeater.cc:503] Master 127.0.236.126:35729 was elected leader, sending a full tablet report...
I20260806 15:03:30.838537 11717 raft_consensus.cc:348] T 164d277c91b8405bb48a91a5fe2c9d9f P 8194526e9ac244b7b939dadfa140b2c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "120512845c1a446db9c5b628af55f0db" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36187 } } peers { permanent_uuid: "8194526e9ac244b7b939dadfa140b2c7" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 35007 } } peers { permanent_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34923 } }
I20260806 15:03:30.838865 11717 raft_consensus.cc:374] T 164d277c91b8405bb48a91a5fe2c9d9f P 8194526e9ac244b7b939dadfa140b2c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:30.838970 11717 raft_consensus.cc:729] T 164d277c91b8405bb48a91a5fe2c9d9f P 8194526e9ac244b7b939dadfa140b2c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8194526e9ac244b7b939dadfa140b2c7, State: Initialized, Role: FOLLOWER
I20260806 15:03:30.839146 11717 consensus_queue.cc:260] T 164d277c91b8405bb48a91a5fe2c9d9f P 8194526e9ac244b7b939dadfa140b2c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "120512845c1a446db9c5b628af55f0db" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36187 } } peers { permanent_uuid: "8194526e9ac244b7b939dadfa140b2c7" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 35007 } } peers { permanent_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34923 } }
I20260806 15:03:30.839443 11717 ts_tablet_manager.cc:1428] T 164d277c91b8405bb48a91a5fe2c9d9f P 8194526e9ac244b7b939dadfa140b2c7: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260806 15:03:30.839509 11672 heartbeater.cc:503] Master 127.0.236.126:35729 was elected leader, sending a full tablet report...
W20260806 15:03:30.899827 11479 tablet.cc:2369] T 164d277c91b8405bb48a91a5fe2c9d9f P b7ccff12a0d44255a60b61e0e3287c5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:03:30.914672 11724 raft_consensus.cc:482] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:03:30.914793 11724 raft_consensus.cc:504] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "120512845c1a446db9c5b628af55f0db" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36187 } } peers { permanent_uuid: "8194526e9ac244b7b939dadfa140b2c7" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 35007 } } peers { permanent_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34923 } }
I20260806 15:03:30.915141 11724 leader_election.cc:288] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8194526e9ac244b7b939dadfa140b2c7 (127.0.236.67:35007), b7ccff12a0d44255a60b61e0e3287c5c (127.0.236.66:34923)
I20260806 15:03:30.917701 11713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.915406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:45901 (local address 127.0.236.67:35007)
0806 15:03:30.915458 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.915460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.915468 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:30.915491 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.915492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.915501 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.915546 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:30.915672 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.916129 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.917057 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.917403 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.917462 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.917471 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.917605 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.917639 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 15:03:30.917708 11290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.915294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:35007 (local address 127.0.236.65:45901)
0806 15:03:30.915398 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:30.915407 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:30.915445 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:30.915554 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:30.915555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:30.915556 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:30.915616 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.915617 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.916196 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.916196 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.916975 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.916977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.917471 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.917472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.917547 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.917550 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:03:30.917594 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 15:03:30.917633 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":40}
I20260806 15:03:30.917735 11716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.915335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:36127 (local address 127.0.236.66:34923)
0806 15:03:30.915375 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.915378 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.915408 (+ 30us) server_negotiation.cc:407] Connection header received
0806 15:03:30.915433 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.915434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.915443 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.915489 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:30.915622 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.916148 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.917058 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.917438 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.917497 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.917506 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.917639 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.917672 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:03:30.917788 11294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.915275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:34923 (local address 127.0.236.65:36127)
0806 15:03:30.915316 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:30.915328 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:30.915373 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:30.915494 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:30.915495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:30.915497 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:30.915562 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.915563 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.916221 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.916222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.916976 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:30.916978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.917502 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.917503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:30.917581 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.917586 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 15:03:30.917628 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 15:03:30.917672 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":2}
I20260806 15:03:30.918066 11408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "164d277c91b8405bb48a91a5fe2c9d9f" candidate_uuid: "120512845c1a446db9c5b628af55f0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" is_pre_election: true
I20260806 15:03:30.918155 11597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "164d277c91b8405bb48a91a5fe2c9d9f" candidate_uuid: "120512845c1a446db9c5b628af55f0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8194526e9ac244b7b939dadfa140b2c7" is_pre_election: true
I20260806 15:03:30.918205 11408 raft_consensus.cc:2454] T 164d277c91b8405bb48a91a5fe2c9d9f P b7ccff12a0d44255a60b61e0e3287c5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 120512845c1a446db9c5b628af55f0db in term 0.
I20260806 15:03:30.918267 11597 raft_consensus.cc:2454] T 164d277c91b8405bb48a91a5fe2c9d9f P 8194526e9ac244b7b939dadfa140b2c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 120512845c1a446db9c5b628af55f0db in term 0.
I20260806 15:03:30.918406 11122 leader_election.cc:302] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 120512845c1a446db9c5b628af55f0db, b7ccff12a0d44255a60b61e0e3287c5c; no voters:
I20260806 15:03:30.918581 11724 raft_consensus.cc:2781] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:03:30.918632 11724 raft_consensus.cc:482] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:03:30.918646 11724 raft_consensus.cc:3037] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:30.919284 11724 raft_consensus.cc:504] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "120512845c1a446db9c5b628af55f0db" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36187 } } peers { permanent_uuid: "8194526e9ac244b7b939dadfa140b2c7" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 35007 } } peers { permanent_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34923 } }
I20260806 15:03:30.919392 11724 leader_election.cc:288] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db [CANDIDATE]: Term 1 election: Requested vote from peers 8194526e9ac244b7b939dadfa140b2c7 (127.0.236.67:35007), b7ccff12a0d44255a60b61e0e3287c5c (127.0.236.66:34923)
I20260806 15:03:30.919526 11408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "164d277c91b8405bb48a91a5fe2c9d9f" candidate_uuid: "120512845c1a446db9c5b628af55f0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ccff12a0d44255a60b61e0e3287c5c"
I20260806 15:03:30.919570 11597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "164d277c91b8405bb48a91a5fe2c9d9f" candidate_uuid: "120512845c1a446db9c5b628af55f0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8194526e9ac244b7b939dadfa140b2c7"
I20260806 15:03:30.919597 11408 raft_consensus.cc:3037] T 164d277c91b8405bb48a91a5fe2c9d9f P b7ccff12a0d44255a60b61e0e3287c5c [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:30.919633 11597 raft_consensus.cc:3037] T 164d277c91b8405bb48a91a5fe2c9d9f P 8194526e9ac244b7b939dadfa140b2c7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:30.920225 11408 raft_consensus.cc:2454] T 164d277c91b8405bb48a91a5fe2c9d9f P b7ccff12a0d44255a60b61e0e3287c5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 120512845c1a446db9c5b628af55f0db in term 1.
I20260806 15:03:30.920315 11597 raft_consensus.cc:2454] T 164d277c91b8405bb48a91a5fe2c9d9f P 8194526e9ac244b7b939dadfa140b2c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 120512845c1a446db9c5b628af55f0db in term 1.
I20260806 15:03:30.920421 11122 leader_election.cc:302] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 120512845c1a446db9c5b628af55f0db, b7ccff12a0d44255a60b61e0e3287c5c; no voters:
I20260806 15:03:30.920514 11724 raft_consensus.cc:2781] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:30.920632 11724 raft_consensus.cc:686] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db [term 1 LEADER]: Becoming Leader. State: Replica: 120512845c1a446db9c5b628af55f0db, State: Running, Role: LEADER
I20260806 15:03:30.920773 11724 consensus_queue.cc:237] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "120512845c1a446db9c5b628af55f0db" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36187 } } peers { permanent_uuid: "8194526e9ac244b7b939dadfa140b2c7" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 35007 } } peers { permanent_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34923 } }
I20260806 15:03:30.921494 9945 catalog_manager.cc:5495] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db reported cstate change: term changed from 0 to 1, leader changed from <none> to 120512845c1a446db9c5b628af55f0db (127.0.236.65). New cstate: current_term: 1 leader_uuid: "120512845c1a446db9c5b628af55f0db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "120512845c1a446db9c5b628af55f0db" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8194526e9ac244b7b939dadfa140b2c7" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 35007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34923 } health_report { overall_health: UNKNOWN } } }
I20260806 15:03:30.939742 11712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.937865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60364 (local address 127.0.236.65:36187)
0806 15:03:30.937952 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.937954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.937962 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:30.937990 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.937990 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.938004 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.938059 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:03:30.938158 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.938588 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.939219 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.939382 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.939420 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.939520 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:03:30.939604 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:03:30.939657 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.939669 (+ 12us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.939684 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 15:03:30.941226 11408 raft_consensus.cc:1261] T 164d277c91b8405bb48a91a5fe2c9d9f P b7ccff12a0d44255a60b61e0e3287c5c [term 1 FOLLOWER]: Refusing update from remote peer 120512845c1a446db9c5b628af55f0db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:30.941232 11597 raft_consensus.cc:1261] T 164d277c91b8405bb48a91a5fe2c9d9f P 8194526e9ac244b7b939dadfa140b2c7 [term 1 FOLLOWER]: Refusing update from remote peer 120512845c1a446db9c5b628af55f0db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:30.941538 11728 consensus_queue.cc:1035] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:30.941650 11724 consensus_queue.cc:1035] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "8194526e9ac244b7b939dadfa140b2c7" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 35007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:30.944195 11732 mvcc.cc:205] Tried to move back new op lower bound from 7315573190413209600 to 7315573190332030976. Current Snapshot: MvccSnapshot[applied={T|T < 7315573190413209600}]
I20260806 15:03:30.944470 11735 mvcc.cc:205] Tried to move back new op lower bound from 7315573190413209600 to 7315573190332030976. Current Snapshot: MvccSnapshot[applied={T|T < 7315573190413209600}]
I20260806 15:03:30.946724 9944 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:30.946805 9944 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:30.947922 9944 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c92fd02274d84e3db9c69443339af692: Sending DeleteTablet for 3 replicas of tablet 164d277c91b8405bb48a91a5fe2c9d9f
I20260806 15:03:30.948201 11183 tablet_service.cc:1514] Processing DeleteTablet for tablet 164d277c91b8405bb48a91a5fe2c9d9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:30 UTC) from {username='oryx', principal='oryx/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:60350
I20260806 15:03:30.948484 11745 tablet_replica.cc:330] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db: stopping tablet replica
I20260806 15:03:30.948485 11383 tablet_service.cc:1514] Processing DeleteTablet for tablet 164d277c91b8405bb48a91a5fe2c9d9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:30 UTC) from {username='oryx', principal='oryx/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:49986
I20260806 15:03:30.948657 11745 raft_consensus.cc:2229] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:03:30.948706 11747 tablet_replica.cc:330] T 164d277c91b8405bb48a91a5fe2c9d9f P b7ccff12a0d44255a60b61e0e3287c5c: stopping tablet replica
I20260806 15:03:30.948839 11747 raft_consensus.cc:2229] T 164d277c91b8405bb48a91a5fe2c9d9f P b7ccff12a0d44255a60b61e0e3287c5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:30.948959 11747 raft_consensus.cc:2258] T 164d277c91b8405bb48a91a5fe2c9d9f P b7ccff12a0d44255a60b61e0e3287c5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:30.949074 11573 tablet_service.cc:1514] Processing DeleteTablet for tablet 164d277c91b8405bb48a91a5fe2c9d9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:30 UTC) from {username='oryx', principal='oryx/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:52666
I20260806 15:03:30.949479 11747 ts_tablet_manager.cc:1905] T 164d277c91b8405bb48a91a5fe2c9d9f P b7ccff12a0d44255a60b61e0e3287c5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:30.948843 11745 raft_consensus.cc:2258] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:30.949690 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 9893
I20260806 15:03:30.956367 11748 tablet_replica.cc:330] T 164d277c91b8405bb48a91a5fe2c9d9f P 8194526e9ac244b7b939dadfa140b2c7: stopping tablet replica
I20260806 15:03:30.956461 11748 raft_consensus.cc:2229] T 164d277c91b8405bb48a91a5fe2c9d9f P 8194526e9ac244b7b939dadfa140b2c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:30.956548 11748 raft_consensus.cc:2258] T 164d277c91b8405bb48a91a5fe2c9d9f P 8194526e9ac244b7b939dadfa140b2c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:30.956950 11748 ts_tablet_manager.cc:1905] T 164d277c91b8405bb48a91a5fe2c9d9f P 8194526e9ac244b7b939dadfa140b2c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:30.958081 11747 ts_tablet_manager.cc:1918] T 164d277c91b8405bb48a91a5fe2c9d9f P b7ccff12a0d44255a60b61e0e3287c5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:03:30.958190 11747 log.cc:1196] T 164d277c91b8405bb48a91a5fe2c9d9f P b7ccff12a0d44255a60b61e0e3287c5c: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/wal/wals/164d277c91b8405bb48a91a5fe2c9d9f
I20260806 15:03:30.958487 11747 ts_tablet_manager.cc:1939] T 164d277c91b8405bb48a91a5fe2c9d9f P b7ccff12a0d44255a60b61e0e3287c5c: Deleting consensus metadata
I20260806 15:03:30.958740 11748 ts_tablet_manager.cc:1918] T 164d277c91b8405bb48a91a5fe2c9d9f P 8194526e9ac244b7b939dadfa140b2c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:03:30.958861 11748 log.cc:1196] T 164d277c91b8405bb48a91a5fe2c9d9f P 8194526e9ac244b7b939dadfa140b2c7: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/164d277c91b8405bb48a91a5fe2c9d9f
I20260806 15:03:30.959204 11748 ts_tablet_manager.cc:1939] T 164d277c91b8405bb48a91a5fe2c9d9f P 8194526e9ac244b7b939dadfa140b2c7: Deleting consensus metadata
I20260806 15:03:30.962275 11745 ts_tablet_manager.cc:1905] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:30.963732 11745 ts_tablet_manager.cc:1918] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:03:30.963805 11745 log.cc:1196] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-0/wal/wals/164d277c91b8405bb48a91a5fe2c9d9f
I20260806 15:03:30.964068 11745 ts_tablet_manager.cc:1939] T 164d277c91b8405bb48a91a5fe2c9d9f P 120512845c1a446db9c5b628af55f0db: Deleting consensus metadata
W20260806 15:03:30.965754 11318 connection.cc:535] server connection from 127.0.0.1:49986 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 15:03:30.965788 11513 connection.cc:535] server connection from 127.0.0.1:52666 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 15:03:30.966028 11122 connection.cc:535] server connection from 127.0.0.1:60350 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 15:03:30.974382 10836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.967505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42298 (local address 127.0.236.125:38713)
0806 15:03:30.967545 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.967547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.967637 (+ 90us) server_negotiation.cc:407] Connection header received
0806 15:03:30.967709 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.967710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.967722 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.967769 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:30.968393 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.969187 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.970174 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.970380 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.970433 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.971964 (+ 1531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:30.971965 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:30.971966 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:30.971973 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:30.973359 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.973658 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.973659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.973660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.973716 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.973878 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.973879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.973880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.974022 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:30.974112 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.974218 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.974297 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 15:03:30.974836 11691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:30.967424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57912 (local address 127.0.236.124:39837)
0806 15:03:30.967463 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:30.967465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:30.967855 (+ 390us) server_negotiation.cc:407] Connection header received
0806 15:03:30.967885 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:30.967886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:30.967900 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:30.967955 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:30.968492 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.969043 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.970145 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:30.970481 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:30.970550 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:30.971940 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:30.971941 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:30.971942 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:30.971949 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:30.973609 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.974013 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.974014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.974015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.974071 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:30.974267 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:30.974268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:30.974268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:30.974390 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:30.974459 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:30.974669 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 15:03:30.974753 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 15:03:31.945995 11760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:31.945665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.65:50177)
0806 15:03:31.945829 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:31.945936 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":67,"threads_started":1}
I20260806 15:03:31.946238 11763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:31.946004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.66:50269)
0806 15:03:31.946130 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:31.946192 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":54,"threads_started":1}
W20260806 15:03:31.946264 11286 heartbeater.cc:644] Failed to heartbeat to 127.0.236.126:35729 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
W20260806 15:03:31.946484 11477 heartbeater.cc:644] Failed to heartbeat to 127.0.236.126:35729 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
I20260806 15:03:31.946856 11764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:31.946651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.67:56769)
0806 15:03:31.946768 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:31.946823 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":56,"threads_started":1}
W20260806 15:03:31.947039 11672 heartbeater.cc:644] Failed to heartbeat to 127.0.236.126:35729 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
I20260806 15:03:32.448230 11765 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c92fd02274d84e3db9c69443339af692)
I20260806 15:03:32.448354 11765 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: false } }
I20260806 15:03:32.448613 11765 leader_election.cc:288] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c92fd02274d84e3db9c69443339af692 (127.0.236.126:35729), 44b909a3dfab4801a2496d6f37cb3d8c (127.0.236.124:39837)
I20260806 15:03:32.448798 11766 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c92fd02274d84e3db9c69443339af692)
I20260806 15:03:32.448905 11767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:32.448705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.0.1:42892)
0806 15:03:32.448813 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:32.448862 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":42,"threads_started":1}
I20260806 15:03:32.448999 11766 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: false } }
W20260806 15:03:32.449064 10748 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
I20260806 15:03:32.449424 11767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:32.449340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.0.1:42910)
0806 15:03:32.449376 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:32.449398 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 15:03:32.449424 11766 leader_election.cc:288] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c92fd02274d84e3db9c69443339af692 (127.0.236.126:35729), e7d2710f37144ab1bd06312edcde3f45 (127.0.236.125:38713)
W20260806 15:03:32.449494 10748 leader_election.cc:334] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c92fd02274d84e3db9c69443339af692 (127.0.236.126:35729): Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
I20260806 15:03:32.449676 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:32.449320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.0.1:42906)
0806 15:03:32.449591 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:32.449641 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":49,"threads_started":1}
W20260806 15:03:32.449816 11000 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
I20260806 15:03:32.450238 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:32.450151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.0.1:42914)
0806 15:03:32.450190 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:32.450214 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 15:03:32.450323 11000 leader_election.cc:334] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c92fd02274d84e3db9c69443339af692 (127.0.236.126:35729): Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
I20260806 15:03:32.451984 11768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:32.448736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:39837 (local address 127.0.0.1:57924)
0806 15:03:32.448907 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:32.448933 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:32.448997 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:32.449178 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:32.449180 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:32.449182 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:32.449239 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:32.449240 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:32.450343 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:32.450344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:32.451254 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:32.451256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:32.451767 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:32.451768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:32.451825 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:32.451828 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:03:32.451871 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 15:03:32.451923 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":55,"threads_started":1}
I20260806 15:03:32.451983 11769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:32.448864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57924 (local address 127.0.236.124:39837)
0806 15:03:32.448942 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:32.448944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:32.449051 (+ 107us) server_negotiation.cc:407] Connection header received
0806 15:03:32.449077 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:32.449078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:32.449089 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:32.449144 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:32.449318 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:32.450234 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:32.451326 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:32.451698 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:32.451756 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:32.451767 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:32.451881 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 15:03:32.451919 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62,"thread_start_us":37,"threads_started":1}
I20260806 15:03:32.452283 11771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:32.449351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:38713 (local address 127.0.0.1:42300)
0806 15:03:32.449607 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:32.449617 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:32.449679 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:32.449828 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:32.449829 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:32.449831 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:32.449880 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:32.449882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:32.450764 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:32.450764 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:32.451524 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:32.451526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:32.452063 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:32.452064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:32.452128 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:32.452132 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:03:32.452176 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 15:03:32.452214 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":44,"threads_started":1}
I20260806 15:03:32.452291 11774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:32.449467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42300 (local address 127.0.236.125:38713)
0806 15:03:32.449590 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:32.449592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:32.449678 (+ 86us) server_negotiation.cc:407] Connection header received
0806 15:03:32.449740 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:32.449741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:32.449751 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:32.449812 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:32.449961 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:32.450694 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:32.451608 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:32.451992 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:32.452049 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:32.452059 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:32.452185 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 15:03:32.452224 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":82,"threads_started":1}
I20260806 15:03:32.452437 11035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e7d2710f37144ab1bd06312edcde3f45" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" is_pre_election: true
I20260806 15:03:32.452543 11035 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7d2710f37144ab1bd06312edcde3f45 in term 1.
I20260806 15:03:32.452514 10777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e7d2710f37144ab1bd06312edcde3f45" is_pre_election: true
I20260806 15:03:32.452592 10777 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44b909a3dfab4801a2496d6f37cb3d8c in term 1.
I20260806 15:03:32.452726 10750 leader_election.cc:302] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 44b909a3dfab4801a2496d6f37cb3d8c, e7d2710f37144ab1bd06312edcde3f45; no voters: c92fd02274d84e3db9c69443339af692
I20260806 15:03:32.452740 10997 leader_election.cc:302] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 44b909a3dfab4801a2496d6f37cb3d8c, e7d2710f37144ab1bd06312edcde3f45; no voters: c92fd02274d84e3db9c69443339af692
I20260806 15:03:32.452831 11765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 15:03:32.452844 11766 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 15:03:32.452870 11766 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [term 1 FOLLOWER]: Starting leader election (detected failure of leader c92fd02274d84e3db9c69443339af692)
I20260806 15:03:32.452873 11765 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c92fd02274d84e3db9c69443339af692)
I20260806 15:03:32.452889 11766 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [term 1 FOLLOWER]: Advancing to term 2
I20260806 15:03:32.452893 11765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 1 FOLLOWER]: Advancing to term 2
I20260806 15:03:32.453543 11765 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: false } }
I20260806 15:03:32.453553 11766 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: false } }
I20260806 15:03:32.453655 11766 leader_election.cc:288] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [CANDIDATE]: Term 2 election: Requested vote from peers c92fd02274d84e3db9c69443339af692 (127.0.236.126:35729), e7d2710f37144ab1bd06312edcde3f45 (127.0.236.125:38713)
I20260806 15:03:32.453655 11765 leader_election.cc:288] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [CANDIDATE]: Term 2 election: Requested vote from peers c92fd02274d84e3db9c69443339af692 (127.0.236.126:35729), 44b909a3dfab4801a2496d6f37cb3d8c (127.0.236.124:39837)
I20260806 15:03:32.453790 11035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e7d2710f37144ab1bd06312edcde3f45" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "44b909a3dfab4801a2496d6f37cb3d8c"
I20260806 15:03:32.453814 10777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e7d2710f37144ab1bd06312edcde3f45"
I20260806 15:03:32.453866 11035 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e7d2710f37144ab1bd06312edcde3f45 in current term 2: Already voted for candidate 44b909a3dfab4801a2496d6f37cb3d8c in this term.
I20260806 15:03:32.453868 11768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:32.453767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.0.1:42928)
0806 15:03:32.453800 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:32.453823 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260806 15:03:32.453917 10777 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 44b909a3dfab4801a2496d6f37cb3d8c in current term 2: Already voted for candidate e7d2710f37144ab1bd06312edcde3f45 in this term.
I20260806 15:03:32.454052 11771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:32.453747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.0.1:42924)
0806 15:03:32.453998 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:32.454029 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236}
I20260806 15:03:32.454298 11768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:32.454208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.0.1:42932)
0806 15:03:32.454249 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:32.454276 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 15:03:32.454363 10748 leader_election.cc:334] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c92fd02274d84e3db9c69443339af692 (127.0.236.126:35729): Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
I20260806 15:03:32.454393 11771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:32.454299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.0.1:42946)
0806 15:03:32.454343 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:32.454370 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 15:03:32.454406 10748 leader_election.cc:302] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7d2710f37144ab1bd06312edcde3f45; no voters: 44b909a3dfab4801a2496d6f37cb3d8c, c92fd02274d84e3db9c69443339af692
W20260806 15:03:32.454452 11000 leader_election.cc:334] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c92fd02274d84e3db9c69443339af692 (127.0.236.126:35729): Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
I20260806 15:03:32.454502 11765 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 15:03:32.454517 11000 leader_election.cc:302] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 44b909a3dfab4801a2496d6f37cb3d8c; no voters: c92fd02274d84e3db9c69443339af692, e7d2710f37144ab1bd06312edcde3f45
I20260806 15:03:32.454578 11766 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 15:03:32.946775 11776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:32.946568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.65:46091)
0806 15:03:32.946697 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:32.946732 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
I20260806 15:03:32.946956 11777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:32.946750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.66:36687)
0806 15:03:32.946862 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:32.946894 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":51,"threads_started":1}
I20260806 15:03:32.947455 11778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:32.947269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.67:42743)
0806 15:03:32.947380 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:32.947422 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
I20260806 15:03:33.947461 11779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:33.947191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.65:48485)
0806 15:03:33.947348 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:33.947393 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":81,"threads_started":1}
I20260806 15:03:33.947604 11780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:33.947392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.66:38377)
0806 15:03:33.947525 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:33.947559 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260806 15:03:33.947971 11785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:33.947802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.67:52265)
0806 15:03:33.947899 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:33.947937 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":48,"threads_started":1}
I20260806 15:03:33.978911 11786 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:03:33.978991 11786 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: false } }
I20260806 15:03:33.979131 11786 leader_election.cc:288] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c92fd02274d84e3db9c69443339af692 (127.0.236.126:35729), 44b909a3dfab4801a2496d6f37cb3d8c (127.0.236.124:39837)
I20260806 15:03:33.979319 11035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e7d2710f37144ab1bd06312edcde3f45" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" is_pre_election: true
I20260806 15:03:33.979388 11787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:33.979215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.0.1:43902)
0806 15:03:33.979321 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:33.979353 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":59,"threads_started":1}
I20260806 15:03:33.979398 11035 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7d2710f37144ab1bd06312edcde3f45 in term 2.
I20260806 15:03:33.979544 10750 leader_election.cc:302] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44b909a3dfab4801a2496d6f37cb3d8c, e7d2710f37144ab1bd06312edcde3f45; no voters:
I20260806 15:03:33.979624 11786 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 15:03:33.979665 11786 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:03:33.979684 11786 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 2 FOLLOWER]: Advancing to term 3
I20260806 15:03:33.979841 11787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:33.979744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.0.1:43904)
0806 15:03:33.979790 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:33.979819 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 15:03:33.979913 10748 leader_election.cc:334] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c92fd02274d84e3db9c69443339af692 (127.0.236.126:35729): Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
I20260806 15:03:33.980460 11786 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: false } }
I20260806 15:03:33.980574 11786 leader_election.cc:288] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [CANDIDATE]: Term 3 election: Requested vote from peers c92fd02274d84e3db9c69443339af692 (127.0.236.126:35729), 44b909a3dfab4801a2496d6f37cb3d8c (127.0.236.124:39837)
I20260806 15:03:33.980731 11787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:33.980650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.0.1:43920)
0806 15:03:33.980686 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:33.980712 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260806 15:03:33.980715 11035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e7d2710f37144ab1bd06312edcde3f45" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "44b909a3dfab4801a2496d6f37cb3d8c"
I20260806 15:03:33.980790 11035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [term 2 FOLLOWER]: Advancing to term 3
I20260806 15:03:33.981392 11035 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7d2710f37144ab1bd06312edcde3f45 in term 3.
I20260806 15:03:33.981550 10750 leader_election.cc:302] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44b909a3dfab4801a2496d6f37cb3d8c, e7d2710f37144ab1bd06312edcde3f45; no voters:
I20260806 15:03:33.981638 11786 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 15:03:33.981693 11787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:33.981611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.0.1:43922)
0806 15:03:33.981649 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:33.981668 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 15:03:33.981737 11786 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [term 3 LEADER]: Becoming Leader. State: Replica: e7d2710f37144ab1bd06312edcde3f45, State: Running, Role: LEADER
W20260806 15:03:33.981779 10748 leader_election.cc:334] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer c92fd02274d84e3db9c69443339af692 (127.0.236.126:35729): Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
I20260806 15:03:33.981846 11786 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: false } }
I20260806 15:03:33.982230 11789 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7d2710f37144ab1bd06312edcde3f45. Latest consensus state: current_term: 3 leader_uuid: "e7d2710f37144ab1bd06312edcde3f45" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: false } } }
I20260806 15:03:33.982319 11789 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:33.982508 11791 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:03:33.983011 11791 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:03:33.983088 11791 catalog_manager.cc:1259] Loaded cluster ID: fef1eb67d42c483abc65fd8ff931c254
I20260806 15:03:33.983119 11791 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:03:33.983294 11791 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:03:33.983363 11791 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45: Loaded TSK: 0
I20260806 15:03:33.983508 11791 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:03:34.038908 10763 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42298:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:03:34.040508 10763 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:03:34.041468 11787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.041366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.0.1:43940)
0806 15:03:34.041411 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:34.041445 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 15:03:34.041487 11035 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [term 3 FOLLOWER]: Refusing update from remote peer e7d2710f37144ab1bd06312edcde3f45: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260806 15:03:34.041754 11787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.041678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.0.1:43942)
0806 15:03:34.041709 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:34.041736 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260806 15:03:34.041711 11789 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260806 15:03:34.041831 10748 consensus_peers.cc:487] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 -> Peer c92fd02274d84e3db9c69443339af692 (127.0.236.126:35729): Couldn't send request to peer c92fd02274d84e3db9c69443339af692. Status: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 15:03:34.042325 11793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7d2710f37144ab1bd06312edcde3f45. Latest consensus state: current_term: 3 leader_uuid: "e7d2710f37144ab1bd06312edcde3f45" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: false } } }
I20260806 15:03:34.042412 11793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:03:34.042807 11786 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "e7d2710f37144ab1bd06312edcde3f45" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: false } } }
I20260806 15:03:34.042888 11786 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:34.042981 11793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "e7d2710f37144ab1bd06312edcde3f45" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c92fd02274d84e3db9c69443339af692" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35729 } } peers { permanent_uuid: "e7d2710f37144ab1bd06312edcde3f45" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 38713 } attrs { promote: false } } peers { permanent_uuid: "44b909a3dfab4801a2496d6f37cb3d8c" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 39837 } attrs { promote: false } } }
I20260806 15:03:34.043062 11793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44b909a3dfab4801a2496d6f37cb3d8c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:03:34.047837 11805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.044678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55848 (local address 127.0.236.67:35007)
0806 15:03:34.044787 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:34.044789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:34.044797 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:34.044821 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:34.044822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:34.044832 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:34.045136 (+ 304us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:34.045231 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:34.045978 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:34.047160 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:34.047535 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:34.047593 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:34.047606 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:34.047738 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 15:03:34.047771 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":61,"threads_started":1}
I20260806 15:03:34.047847 11801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.044513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:35007 (local address 127.0.0.1:55848)
0806 15:03:34.044637 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:34.044645 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:34.044682 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:34.045150 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:34.045151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:34.045152 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:34.045180 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:34.045181 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:34.046064 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:34.046065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:34.047097 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:34.047099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:34.047608 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:34.047609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:34.047680 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:34.047685 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 15:03:34.047728 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 15:03:34.047771 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260806 15:03:34.047976 11802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.044463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:36187 (local address 127.0.0.1:47816)
0806 15:03:34.044607 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:34.044616 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:34.044655 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:34.045066 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:34.045067 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:34.045069 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:34.045097 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:34.045098 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:34.045812 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:34.045813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:34.046995 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:34.046998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:34.047771 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:34.047772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:34.047837 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:34.047841 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:03:34.047866 (+ 25us) client_negotiation.cc:240] Negotiation successful
0806 15:03:34.047908 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":73,"threads_started":1}
I20260806 15:03:34.048286 11804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.044679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47816 (local address 127.0.236.65:36187)
0806 15:03:34.044960 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:34.044962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:34.044969 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:03:34.044994 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:34.044994 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:34.045004 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:34.045066 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:34.045257 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:34.045725 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:34.047090 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:34.047679 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:34.048133 (+ 454us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:34.048152 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:34.048171 (+ 19us) server_negotiation.cc:299] Negotiation successful
0806 15:03:34.048212 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":57,"threads_started":1}
I20260806 15:03:34.048367 11573 tablet_service.cc:1467] Processing CreateTablet for tablet b92f06ff7b9d45ef88f24f255a706721 (DEFAULT_TABLE table=test-table [id=bf6676bfa9c14f58a9888889490d447a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:34.048496 11573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b92f06ff7b9d45ef88f24f255a706721. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:34.048496 11183 tablet_service.cc:1467] Processing CreateTablet for tablet b92f06ff7b9d45ef88f24f255a706721 (DEFAULT_TABLE table=test-table [id=bf6676bfa9c14f58a9888889490d447a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:34.048614 11183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b92f06ff7b9d45ef88f24f255a706721. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:34.049160 11787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.044427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:34923 (local address 127.0.0.1:58604)
0806 15:03:34.044455 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:34.044466 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:34.044516 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:34.044873 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:34.044875 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:34.044877 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:34.044929 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:34.044930 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:34.047170 (+ 2240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:34.047170 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:34.048163 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:34.048166 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:34.048927 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:34.048928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:34.049002 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:34.049005 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:03:34.049065 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 15:03:34.049102 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260806 15:03:34.049191 11803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.044525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58604 (local address 127.0.236.66:34923)
0806 15:03:34.044733 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:34.044735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:34.044743 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:34.044768 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:34.044769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:34.044779 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:34.044851 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:34.045156 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:34.045987 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:34.048334 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:34.048838 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:34.048907 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:34.048918 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:34.049077 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 15:03:34.049117 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":63,"threads_started":1}
I20260806 15:03:34.049818 11807 tablet_bootstrap.cc:492] T b92f06ff7b9d45ef88f24f255a706721 P 120512845c1a446db9c5b628af55f0db: Bootstrap starting.
I20260806 15:03:34.050129 11383 tablet_service.cc:1467] Processing CreateTablet for tablet b92f06ff7b9d45ef88f24f255a706721 (DEFAULT_TABLE table=test-table [id=bf6676bfa9c14f58a9888889490d447a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:34.050164 11809 tablet_bootstrap.cc:492] T b92f06ff7b9d45ef88f24f255a706721 P 8194526e9ac244b7b939dadfa140b2c7: Bootstrap starting.
I20260806 15:03:34.050240 11383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b92f06ff7b9d45ef88f24f255a706721. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:34.050277 11807 tablet_bootstrap.cc:654] T b92f06ff7b9d45ef88f24f255a706721 P 120512845c1a446db9c5b628af55f0db: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:34.050480 11809 tablet_bootstrap.cc:654] T b92f06ff7b9d45ef88f24f255a706721 P 8194526e9ac244b7b939dadfa140b2c7: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:34.051023 11809 tablet_bootstrap.cc:492] T b92f06ff7b9d45ef88f24f255a706721 P 8194526e9ac244b7b939dadfa140b2c7: No bootstrap required, opened a new log
I20260806 15:03:34.051064 11809 ts_tablet_manager.cc:1397] T b92f06ff7b9d45ef88f24f255a706721 P 8194526e9ac244b7b939dadfa140b2c7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:34.051172 11809 raft_consensus.cc:348] T b92f06ff7b9d45ef88f24f255a706721 P 8194526e9ac244b7b939dadfa140b2c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34923 } } peers { permanent_uuid: "120512845c1a446db9c5b628af55f0db" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36187 } } peers { permanent_uuid: "8194526e9ac244b7b939dadfa140b2c7" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 35007 } }
I20260806 15:03:34.051223 11809 raft_consensus.cc:374] T b92f06ff7b9d45ef88f24f255a706721 P 8194526e9ac244b7b939dadfa140b2c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:34.051241 11809 raft_consensus.cc:729] T b92f06ff7b9d45ef88f24f255a706721 P 8194526e9ac244b7b939dadfa140b2c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8194526e9ac244b7b939dadfa140b2c7, State: Initialized, Role: FOLLOWER
I20260806 15:03:34.051282 11809 consensus_queue.cc:260] T b92f06ff7b9d45ef88f24f255a706721 P 8194526e9ac244b7b939dadfa140b2c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34923 } } peers { permanent_uuid: "120512845c1a446db9c5b628af55f0db" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36187 } } peers { permanent_uuid: "8194526e9ac244b7b939dadfa140b2c7" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 35007 } }
I20260806 15:03:34.051465 11809 ts_tablet_manager.cc:1428] T b92f06ff7b9d45ef88f24f255a706721 P 8194526e9ac244b7b939dadfa140b2c7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:34.051863 11785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.051778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.67:46915)
0806 15:03:34.051815 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:34.051838 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 15:03:34.052187 11785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.052105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.67:36185)
0806 15:03:34.052137 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:34.052161 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260806 15:03:34.052233 11674 heartbeater.cc:503] Master 127.0.236.125:38713 was elected leader, sending a full tablet report...
I20260806 15:03:34.052263 11815 tablet_bootstrap.cc:492] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c: Bootstrap starting.
I20260806 15:03:34.052762 11815 tablet_bootstrap.cc:654] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:34.053225 11807 tablet_bootstrap.cc:492] T b92f06ff7b9d45ef88f24f255a706721 P 120512845c1a446db9c5b628af55f0db: No bootstrap required, opened a new log
I20260806 15:03:34.053225 11815 tablet_bootstrap.cc:492] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c: No bootstrap required, opened a new log
I20260806 15:03:34.053278 11815 ts_tablet_manager.cc:1397] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:34.053278 11807 ts_tablet_manager.cc:1397] T b92f06ff7b9d45ef88f24f255a706721 P 120512845c1a446db9c5b628af55f0db: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 15:03:34.053395 11815 raft_consensus.cc:348] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34923 } } peers { permanent_uuid: "120512845c1a446db9c5b628af55f0db" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36187 } } peers { permanent_uuid: "8194526e9ac244b7b939dadfa140b2c7" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 35007 } }
I20260806 15:03:34.053398 11807 raft_consensus.cc:348] T b92f06ff7b9d45ef88f24f255a706721 P 120512845c1a446db9c5b628af55f0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34923 } } peers { permanent_uuid: "120512845c1a446db9c5b628af55f0db" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36187 } } peers { permanent_uuid: "8194526e9ac244b7b939dadfa140b2c7" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 35007 } }
I20260806 15:03:34.053462 11807 raft_consensus.cc:374] T b92f06ff7b9d45ef88f24f255a706721 P 120512845c1a446db9c5b628af55f0db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:34.053462 11815 raft_consensus.cc:374] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:34.053483 11807 raft_consensus.cc:729] T b92f06ff7b9d45ef88f24f255a706721 P 120512845c1a446db9c5b628af55f0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 120512845c1a446db9c5b628af55f0db, State: Initialized, Role: FOLLOWER
I20260806 15:03:34.053483 11815 raft_consensus.cc:729] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7ccff12a0d44255a60b61e0e3287c5c, State: Initialized, Role: FOLLOWER
I20260806 15:03:34.053529 11807 consensus_queue.cc:260] T b92f06ff7b9d45ef88f24f255a706721 P 120512845c1a446db9c5b628af55f0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34923 } } peers { permanent_uuid: "120512845c1a446db9c5b628af55f0db" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36187 } } peers { permanent_uuid: "8194526e9ac244b7b939dadfa140b2c7" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 35007 } }
I20260806 15:03:34.053550 11815 consensus_queue.cc:260] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34923 } } peers { permanent_uuid: "120512845c1a446db9c5b628af55f0db" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36187 } } peers { permanent_uuid: "8194526e9ac244b7b939dadfa140b2c7" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 35007 } }
I20260806 15:03:34.053700 11807 ts_tablet_manager.cc:1428] T b92f06ff7b9d45ef88f24f255a706721 P 120512845c1a446db9c5b628af55f0db: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:34.054190 11779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.054006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.65:41251)
0806 15:03:34.054130 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:34.054160 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260806 15:03:34.054288 11288 heartbeater.cc:503] Master 127.0.236.125:38713 was elected leader, sending a full tablet report...
I20260806 15:03:34.054714 11780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.054626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.66:47747)
0806 15:03:34.054657 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:34.054688 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260806 15:03:34.054811 11815 ts_tablet_manager.cc:1428] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:34.054873 11779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.054617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.65:43999)
0806 15:03:34.054820 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:34.054848 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186}
I20260806 15:03:34.055023 11478 heartbeater.cc:503] Master 127.0.236.125:38713 was elected leader, sending a full tablet report...
I20260806 15:03:34.055193 11780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.055109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.66:52633)
0806 15:03:34.055145 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:34.055168 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 15:03:34.254518 11289 tablet.cc:2369] T b92f06ff7b9d45ef88f24f255a706721 P 120512845c1a446db9c5b628af55f0db: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 15:03:34.299638 11676 tablet.cc:2369] T b92f06ff7b9d45ef88f24f255a706721 P 8194526e9ac244b7b939dadfa140b2c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:03:34.433238 11818 raft_consensus.cc:482] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:03:34.433343 11818 raft_consensus.cc:504] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34923 } } peers { permanent_uuid: "120512845c1a446db9c5b628af55f0db" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36187 } } peers { permanent_uuid: "8194526e9ac244b7b939dadfa140b2c7" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 35007 } }
I20260806 15:03:34.433692 11818 leader_election.cc:288] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 120512845c1a446db9c5b628af55f0db (127.0.236.65:36187), 8194526e9ac244b7b939dadfa140b2c7 (127.0.236.67:35007)
I20260806 15:03:34.436405 11780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.433851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:35007 (local address 127.0.236.66:40701)
0806 15:03:34.433924 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:34.433935 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:34.433999 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:34.434145 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:34.434146 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:34.434149 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:34.434207 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:34.434209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:34.434848 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:34.434849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:34.435665 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:34.435668 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:34.436184 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:34.436185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:34.436246 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:34.436250 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:03:34.436295 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 15:03:34.436331 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 15:03:34.436405 11805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.433940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:40701 (local address 127.0.236.67:35007)
0806 15:03:34.434003 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:34.434004 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:34.434029 (+ 25us) server_negotiation.cc:407] Connection header received
0806 15:03:34.434082 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:34.434083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:34.434096 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:34.434140 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:34.434291 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:34.434776 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:34.435750 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:34.436116 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:34.436177 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:34.436187 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:34.436306 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 15:03:34.436342 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 15:03:34.436506 11804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.433947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:55443 (local address 127.0.236.65:36187)
0806 15:03:34.433994 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:34.433996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:34.434151 (+ 155us) server_negotiation.cc:407] Connection header received
0806 15:03:34.434204 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:34.434206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:34.434215 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:34.434262 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:34.434392 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:34.434862 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:34.435813 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:34.436172 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:34.436232 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:34.436241 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:34.436396 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 15:03:34.436441 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 15:03:34.436522 11821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.433846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:36187 (local address 127.0.236.66:55443)
0806 15:03:34.434071 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:34.434083 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:34.434149 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:34.434274 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:34.434275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:34.434276 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:34.434335 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:34.434337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:34.434946 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:34.434946 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:34.435725 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:34.435727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:34.436269 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:34.436270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:34.436330 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:34.436333 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:03:34.436377 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 15:03:34.436414 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":1,"thread_start_us":132,"threads_started":1}
I20260806 15:03:34.436758 11218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b92f06ff7b9d45ef88f24f255a706721" candidate_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "120512845c1a446db9c5b628af55f0db" is_pre_election: true
I20260806 15:03:34.436843 11597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b92f06ff7b9d45ef88f24f255a706721" candidate_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8194526e9ac244b7b939dadfa140b2c7" is_pre_election: true
I20260806 15:03:34.436916 11597 raft_consensus.cc:2454] T b92f06ff7b9d45ef88f24f255a706721 P 8194526e9ac244b7b939dadfa140b2c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7ccff12a0d44255a60b61e0e3287c5c in term 0.
I20260806 15:03:34.436908 11218 raft_consensus.cc:2454] T b92f06ff7b9d45ef88f24f255a706721 P 120512845c1a446db9c5b628af55f0db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7ccff12a0d44255a60b61e0e3287c5c in term 0.
I20260806 15:03:34.437080 11319 leader_election.cc:302] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 120512845c1a446db9c5b628af55f0db, b7ccff12a0d44255a60b61e0e3287c5c; no voters:
I20260806 15:03:34.437192 11818 raft_consensus.cc:2781] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:03:34.437232 11818 raft_consensus.cc:482] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:03:34.437256 11818 raft_consensus.cc:3037] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:34.437968 11818 raft_consensus.cc:504] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34923 } } peers { permanent_uuid: "120512845c1a446db9c5b628af55f0db" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36187 } } peers { permanent_uuid: "8194526e9ac244b7b939dadfa140b2c7" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 35007 } }
I20260806 15:03:34.438088 11818 leader_election.cc:288] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c [CANDIDATE]: Term 1 election: Requested vote from peers 120512845c1a446db9c5b628af55f0db (127.0.236.65:36187), 8194526e9ac244b7b939dadfa140b2c7 (127.0.236.67:35007)
I20260806 15:03:34.438232 11597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b92f06ff7b9d45ef88f24f255a706721" candidate_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8194526e9ac244b7b939dadfa140b2c7"
I20260806 15:03:34.438232 11218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b92f06ff7b9d45ef88f24f255a706721" candidate_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "120512845c1a446db9c5b628af55f0db"
I20260806 15:03:34.438301 11218 raft_consensus.cc:3037] T b92f06ff7b9d45ef88f24f255a706721 P 120512845c1a446db9c5b628af55f0db [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:34.438301 11597 raft_consensus.cc:3037] T b92f06ff7b9d45ef88f24f255a706721 P 8194526e9ac244b7b939dadfa140b2c7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:34.439042 11218 raft_consensus.cc:2454] T b92f06ff7b9d45ef88f24f255a706721 P 120512845c1a446db9c5b628af55f0db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7ccff12a0d44255a60b61e0e3287c5c in term 1.
I20260806 15:03:34.439042 11597 raft_consensus.cc:2454] T b92f06ff7b9d45ef88f24f255a706721 P 8194526e9ac244b7b939dadfa140b2c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7ccff12a0d44255a60b61e0e3287c5c in term 1.
I20260806 15:03:34.439199 11319 leader_election.cc:302] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 120512845c1a446db9c5b628af55f0db, b7ccff12a0d44255a60b61e0e3287c5c; no voters:
I20260806 15:03:34.439273 11818 raft_consensus.cc:2781] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:34.439409 11818 raft_consensus.cc:686] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c [term 1 LEADER]: Becoming Leader. State: Replica: b7ccff12a0d44255a60b61e0e3287c5c, State: Running, Role: LEADER
I20260806 15:03:34.439699 11818 consensus_queue.cc:237] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34923 } } peers { permanent_uuid: "120512845c1a446db9c5b628af55f0db" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36187 } } peers { permanent_uuid: "8194526e9ac244b7b939dadfa140b2c7" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 35007 } }
I20260806 15:03:34.439757 11821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.439670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.66:55027)
0806 15:03:34.439707 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:34.439736 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 15:03:34.440428 10762 catalog_manager.cc:5495] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c reported cstate change: term changed from 0 to 1, leader changed from <none> to b7ccff12a0d44255a60b61e0e3287c5c (127.0.236.66). New cstate: current_term: 1 leader_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ccff12a0d44255a60b61e0e3287c5c" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "120512845c1a446db9c5b628af55f0db" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8194526e9ac244b7b939dadfa140b2c7" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 35007 } health_report { overall_health: UNKNOWN } } }
I20260806 15:03:34.486611 11803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.484408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58608 (local address 127.0.236.66:34923)
0806 15:03:34.484479 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:34.484481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:34.484488 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:03:34.484523 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:34.484524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:34.484542 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:34.484603 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:03:34.484725 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:34.485190 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:34.485919 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:34.486110 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:34.486163 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:34.486298 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:03:34.486400 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:03:34.486470 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:34.486516 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 15:03:34.486539 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 15:03:34.487632 11597 raft_consensus.cc:1261] T b92f06ff7b9d45ef88f24f255a706721 P 8194526e9ac244b7b939dadfa140b2c7 [term 1 FOLLOWER]: Refusing update from remote peer b7ccff12a0d44255a60b61e0e3287c5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:34.487710 11218 raft_consensus.cc:1261] T b92f06ff7b9d45ef88f24f255a706721 P 120512845c1a446db9c5b628af55f0db [term 1 FOLLOWER]: Refusing update from remote peer b7ccff12a0d44255a60b61e0e3287c5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:34.487905 11822 consensus_queue.cc:1035] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8194526e9ac244b7b939dadfa140b2c7" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 35007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:34.488039 11818 consensus_queue.cc:1035] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "120512845c1a446db9c5b628af55f0db" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:34.489192 11779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.489014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.65:35523)
0806 15:03:34.489125 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:34.489159 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260806 15:03:34.489275 11785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.489173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.67:59647)
0806 15:03:34.489218 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:34.489245 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 15:03:34.489725 11828 mvcc.cc:205] Tried to move back new op lower bound from 7315573204939362304 to 7315573204745482240. Current Snapshot: MvccSnapshot[applied={T|T < 7315573204939362304}]
I20260806 15:03:34.490152 11785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.489995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.67:40627)
0806 15:03:34.490096 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:34.490124 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 15:03:34.490391 11821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.490168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.66:36399)
0806 15:03:34.490336 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:34.490365 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260806 15:03:34.490774 11779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.490684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.65:35617)
0806 15:03:34.490723 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:34.490747 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 15:03:34.491246 11821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.491167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.236.66:40207)
0806 15:03:34.491200 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:34.491222 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260806 15:03:34.492398 10762 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:34.492489 10762 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:34.493656 10762 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e7d2710f37144ab1bd06312edcde3f45: Sending DeleteTablet for 3 replicas of tablet b92f06ff7b9d45ef88f24f255a706721
I20260806 15:03:34.493963 11183 tablet_service.cc:1514] Processing DeleteTablet for tablet b92f06ff7b9d45ef88f24f255a706721 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:34 UTC) from {username='oryx', principal='oryx/127.0.236.125@KRBTEST.COM'} at 127.0.0.1:47816
I20260806 15:03:34.494016 11573 tablet_service.cc:1514] Processing DeleteTablet for tablet b92f06ff7b9d45ef88f24f255a706721 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:34 UTC) from {username='oryx', principal='oryx/127.0.236.125@KRBTEST.COM'} at 127.0.0.1:55848
I20260806 15:03:34.494131 11841 tablet_replica.cc:330] T b92f06ff7b9d45ef88f24f255a706721 P 120512845c1a446db9c5b628af55f0db: stopping tablet replica
I20260806 15:03:34.494175 11841 raft_consensus.cc:2229] T b92f06ff7b9d45ef88f24f255a706721 P 120512845c1a446db9c5b628af55f0db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:34.494211 11841 raft_consensus.cc:2258] T b92f06ff7b9d45ef88f24f255a706721 P 120512845c1a446db9c5b628af55f0db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:34.494194 11383 tablet_service.cc:1514] Processing DeleteTablet for tablet b92f06ff7b9d45ef88f24f255a706721 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:34 UTC) from {username='oryx', principal='oryx/127.0.236.125@KRBTEST.COM'} at 127.0.0.1:58604
I20260806 15:03:34.494367 11844 tablet_replica.cc:330] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c: stopping tablet replica
I20260806 15:03:34.494408 11844 raft_consensus.cc:2229] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:03:34.494477 11844 raft_consensus.cc:2258] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:34.494726 11844 ts_tablet_manager.cc:1905] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:34.494819 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 11091
I20260806 15:03:34.497653 11844 ts_tablet_manager.cc:1918] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:03:34.497696 11844 log.cc:1196] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-1/wal/wals/b92f06ff7b9d45ef88f24f255a706721
I20260806 15:03:34.497869 11844 ts_tablet_manager.cc:1939] T b92f06ff7b9d45ef88f24f255a706721 P b7ccff12a0d44255a60b61e0e3287c5c: Deleting consensus metadata
I20260806 15:03:34.498126 10750 catalog_manager.cc:4841] TS b7ccff12a0d44255a60b61e0e3287c5c (127.0.236.66:34923): tablet b92f06ff7b9d45ef88f24f255a706721 (table test-table [id=bf6676bfa9c14f58a9888889490d447a]) successfully deleted
I20260806 15:03:34.505196 11842 tablet_replica.cc:330] T b92f06ff7b9d45ef88f24f255a706721 P 8194526e9ac244b7b939dadfa140b2c7: stopping tablet replica
I20260806 15:03:34.505255 11842 raft_consensus.cc:2229] T b92f06ff7b9d45ef88f24f255a706721 P 8194526e9ac244b7b939dadfa140b2c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:34.505336 11842 raft_consensus.cc:2258] T b92f06ff7b9d45ef88f24f255a706721 P 8194526e9ac244b7b939dadfa140b2c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:34.505590 11842 ts_tablet_manager.cc:1905] T b92f06ff7b9d45ef88f24f255a706721 P 8194526e9ac244b7b939dadfa140b2c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:34.506906 11842 ts_tablet_manager.cc:1918] T b92f06ff7b9d45ef88f24f255a706721 P 8194526e9ac244b7b939dadfa140b2c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:03:34.506965 11842 log.cc:1196] T b92f06ff7b9d45ef88f24f255a706721 P 8194526e9ac244b7b939dadfa140b2c7: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/b92f06ff7b9d45ef88f24f255a706721
I20260806 15:03:34.507174 11842 ts_tablet_manager.cc:1939] T b92f06ff7b9d45ef88f24f255a706721 P 8194526e9ac244b7b939dadfa140b2c7: Deleting consensus metadata
I20260806 15:03:34.507450 10749 catalog_manager.cc:4841] TS 8194526e9ac244b7b939dadfa140b2c7 (127.0.236.67:35007): tablet b92f06ff7b9d45ef88f24f255a706721 (table test-table [id=bf6676bfa9c14f58a9888889490d447a]) successfully deleted
W20260806 15:03:34.512987 11787 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:34.512876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:36187 (local address 127.0.0.1:47822)
0806 15:03:34.512917 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:34.512942 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.65:36187: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 15:03:34.513099 10748 catalog_manager.cc:4584] TS 120512845c1a446db9c5b628af55f0db (127.0.236.65:36187): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b92f06ff7b9d45ef88f24f255a706721: Network error: Client connection negotiation failed: client connection to 127.0.236.65:36187: connect: Connection reset by peer (error 104)
I20260806 15:03:34.513223 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 11295
I20260806 15:03:34.520807 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 11483
I20260806 15:03:34.529104 11787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.528983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.0.1:43944)
0806 15:03:34.529031 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:34.529072 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 15:03:34.529412 11787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:34.529324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35729 (local address 127.0.0.1:43950)
0806 15:03:34.529362 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:34.529389 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:35729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 15:03:34.529749 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 10594
I20260806 15:03:34.542229 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 10840
2026-08-06T15:03:34Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (8213 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:03:34 dist-test-slave-tc2s krb5kdc[11848](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:03:34 dist-test-slave-tc2s krb5kdc[11848](info): set up 2 sockets
Aug 06 15:03:34 dist-test-slave-tc2s krb5kdc[11848](info): commencing operation
krb5kdc: starting...
W20260806 15:03:35.654186 945 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:03:35 dist-test-slave-tc2s krb5kdc[11848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:03:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:03:35Z Disabled control of system clock
WARNING: no policy specified for oryx/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/oryx_127.0.236.126.keytab.
Entry for principal oryx/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/oryx_127.0.236.126.keytab.
I20260806 15:03:35.762458 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:39739
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/oryx_127.0.236.126.keytab
--principal=oryx/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:42293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:39739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:35.840359 11868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:35.840548 11868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:35.840569 11868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:35.841779 11868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:03:35.841822 11868 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 15:03:35.841835 11868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:35.841847 11868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:35.841871 11868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:03:35.841917 11868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:03:35.843379 11868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:42293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:39739
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/oryx_127.0.236.126.keytab
--principal=oryx/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:39739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:35.843598 11868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:35.843796 11868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:35.848256 11879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:35.848256 11882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:35.848284 11868 server_base.cc:1034] running on GCE node
W20260806 15:03:35.848282 11880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:35.848716 11868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:35.849002 11868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:35.850136 11868 hybrid_clock.cc:648] HybridClock initialized: now 1786028615850120 us; error 29 us; skew 500 ppm
Aug 06 15:03:35 dist-test-slave-tc2s krb5kdc[11848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028615, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:35.852332 11868 init.cc:377] Logged in from keytab as oryx/127.0.236.126@KRBTEST.COM (short username oryx)
I20260806 15:03:35.852751 11868 webserver.cc:460] Webserver started at http://127.0.236.126:39001/ using document root <none> and password file <none>
I20260806 15:03:35.852944 11868 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:35.852993 11868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:35.853116 11868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:35.854100 11868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "366abdf501094dc593ca43cdbfc488ef"
format_stamp: "Formatted at 2026-08-06 15:03:35 on dist-test-slave-tc2s"
server_key: "dd0fd5676ee8ae4ca6ef95e43ef4edd1"
server_key_iv: "ff64ad4ebbd8e32d6aaeca4a1f920201"
server_key_version: "encryptionkey@0"
I20260806 15:03:35.854406 11868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "366abdf501094dc593ca43cdbfc488ef"
format_stamp: "Formatted at 2026-08-06 15:03:35 on dist-test-slave-tc2s"
server_key: "dd0fd5676ee8ae4ca6ef95e43ef4edd1"
server_key_iv: "ff64ad4ebbd8e32d6aaeca4a1f920201"
server_key_version: "encryptionkey@0"
I20260806 15:03:35.855581 11868 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:35.856228 11893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:35.856369 11868 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:35.856451 11868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "366abdf501094dc593ca43cdbfc488ef"
format_stamp: "Formatted at 2026-08-06 15:03:35 on dist-test-slave-tc2s"
server_key: "dd0fd5676ee8ae4ca6ef95e43ef4edd1"
server_key_iv: "ff64ad4ebbd8e32d6aaeca4a1f920201"
server_key_version: "encryptionkey@0"
I20260806 15:03:35.856510 11868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:35.876113 11868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:35.877650 11868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:35.877776 11868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:35.882126 11868 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:39739
I20260806 15:03:35.882193 11983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:39739 every 8 connection(s)
I20260806 15:03:35.882457 11868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:03:35.883018 11984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:35.885077 11984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef: Bootstrap starting.
I20260806 15:03:35.885615 11984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:35.885901 11984 log.cc:824] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:35.886235 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 11868
I20260806 15:03:35.886339 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:03:35.886549 945 external_mini_cluster.cc:1428] Setting key f725ff4d44c284668cc5bfce14dec7fb
I20260806 15:03:35.886593 11984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef: No bootstrap required, opened a new log
I20260806 15:03:35.889143 11984 raft_consensus.cc:348] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "366abdf501094dc593ca43cdbfc488ef" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39739 } }
I20260806 15:03:35.889281 11984 raft_consensus.cc:374] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:35.889303 11984 raft_consensus.cc:729] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 366abdf501094dc593ca43cdbfc488ef, State: Initialized, Role: FOLLOWER
I20260806 15:03:35.889406 11984 consensus_queue.cc:260] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "366abdf501094dc593ca43cdbfc488ef" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39739 } }
I20260806 15:03:35.889477 11984 raft_consensus.cc:388] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:03:35.889501 11984 raft_consensus.cc:482] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:03:35.889544 11984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:35.890143 11984 raft_consensus.cc:504] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "366abdf501094dc593ca43cdbfc488ef" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39739 } }
I20260806 15:03:35.890251 11984 leader_election.cc:302] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 366abdf501094dc593ca43cdbfc488ef; no voters:
I20260806 15:03:35.890398 11984 leader_election.cc:288] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:03:35.890430 11992 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:35.890587 11992 raft_consensus.cc:686] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef [term 1 LEADER]: Becoming Leader. State: Replica: 366abdf501094dc593ca43cdbfc488ef, State: Running, Role: LEADER
I20260806 15:03:35.890607 11984 sys_catalog.cc:564] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:03:35.891602 11992 consensus_queue.cc:237] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "366abdf501094dc593ca43cdbfc488ef" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39739 } }
I20260806 15:03:35.892321 11994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 366abdf501094dc593ca43cdbfc488ef. Latest consensus state: current_term: 1 leader_uuid: "366abdf501094dc593ca43cdbfc488ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "366abdf501094dc593ca43cdbfc488ef" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39739 } } }
I20260806 15:03:35.892407 11994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:35.892583 11995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "366abdf501094dc593ca43cdbfc488ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "366abdf501094dc593ca43cdbfc488ef" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39739 } } }
I20260806 15:03:35.893050 11995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef [sys.catalog]: This master's current role is: LEADER
W20260806 15:03:35.892665 12017 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:03:35.893147 12017 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:03:35.893229 12021 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:03:35.893416 12021 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 15:03:35 dist-test-slave-tc2s krb5kdc[11848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM
I20260806 15:03:35.894972 12021 catalog_manager.cc:1347] Generated new cluster ID: 515cc1bb24d94456a1a2ff588808abf5
I20260806 15:03:35.895026 12021 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:03:35.904107 11991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:35.887051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38964 (local address 127.0.236.126:39739)
0806 15:03:35.887209 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:35.887217 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:35.887240 (+ 23us) server_negotiation.cc:407] Connection header received
0806 15:03:35.887839 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:35.887845 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:35.887888 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:35.887997 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:35.888765 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:35.889770 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:35.890807 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:35.891295 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:35.891386 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:35.893940 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:35.893945 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:35.893945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:35.893957 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:35.902532 (+ 8575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:35.902892 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:35.902896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:35.902896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:35.903021 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:35.903198 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:35.903198 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:35.903199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:35.903499 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:35.903633 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:35.903726 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 15:03:35.903863 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
I20260806 15:03:35.919163 12021 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:03:35.919777 12021 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:03:35.928478 12021 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef: Generated new TSK 0
I20260806 15:03:35.928635 12021 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.0.236.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.236.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.236.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/oryx_127.0.236.65.keytab.
Entry for principal oryx/127.0.236.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/oryx_127.0.236.65.keytab.
I20260806 15:03:35.956525 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:0
--local_ip_for_outbound_sockets=127.0.236.65
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:39739
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/oryx_127.0.236.65.keytab
--principal=oryx/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:42293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:36.039304 12029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:36.039492 12029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:36.039515 12029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:36.040634 12029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:36.040674 12029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:36.040694 12029 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 15:03:36.040722 12029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:03:36.042284 12029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:42293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/oryx_127.0.236.65.keytab
--principal=oryx/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.236.126:39739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:36.042487 12029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:36.042697 12029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:36.047212 12039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:36.047194 12040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:36.047257 12029 server_base.cc:1034] running on GCE node
W20260806 15:03:36.047187 12042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:36.047588 12029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:36.047791 12029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:36.048924 12029 hybrid_clock.cc:648] HybridClock initialized: now 1786028616048910 us; error 33 us; skew 500 ppm
Aug 06 15:03:36 dist-test-slave-tc2s krb5kdc[11848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028616, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:36.051287 12029 init.cc:377] Logged in from keytab as oryx/127.0.236.65@KRBTEST.COM (short username oryx)
I20260806 15:03:36.051671 12029 webserver.cc:460] Webserver started at http://127.0.236.65:46499/ using document root <none> and password file <none>
I20260806 15:03:36.051853 12029 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:36.051899 12029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:36.052022 12029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:36.052947 12029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-0/data/instance:
uuid: "b584a933f35a49678c7f0adde4e4c211"
format_stamp: "Formatted at 2026-08-06 15:03:36 on dist-test-slave-tc2s"
server_key: "f6addfee7e3cb51f5389beb0f45ed3e4"
server_key_iv: "01444fd1c3921ed0d8d5819eb4ce59a0"
server_key_version: "encryptionkey@0"
I20260806 15:03:36.053282 12029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-0/wal/instance:
uuid: "b584a933f35a49678c7f0adde4e4c211"
format_stamp: "Formatted at 2026-08-06 15:03:36 on dist-test-slave-tc2s"
server_key: "f6addfee7e3cb51f5389beb0f45ed3e4"
server_key_iv: "01444fd1c3921ed0d8d5819eb4ce59a0"
server_key_version: "encryptionkey@0"
I20260806 15:03:36.054510 12029 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:03:36.055127 12053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:36.055271 12029 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:03:36.055351 12029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "b584a933f35a49678c7f0adde4e4c211"
format_stamp: "Formatted at 2026-08-06 15:03:36 on dist-test-slave-tc2s"
server_key: "f6addfee7e3cb51f5389beb0f45ed3e4"
server_key_iv: "01444fd1c3921ed0d8d5819eb4ce59a0"
server_key_version: "encryptionkey@0"
I20260806 15:03:36.055419 12029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:36.069705 12029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:36.071702 12029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:36.071832 12029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:36.072266 12029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:36.072304 12029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:36.072328 12029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:36.072345 12029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:36.078688 12029 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:46041
Aug 06 15:03:36 dist-test-slave-tc2s krb5kdc[11848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028616, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM
I20260806 15:03:36.078768 12241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:46041 every 8 connection(s)
I20260806 15:03:36.079049 12029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:03:36.081171 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 12029
I20260806 15:03:36.081287 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-0/wal/instance
I20260806 15:03:36.081470 945 external_mini_cluster.cc:1428] Setting key dc87f5c454169f3579a3949ade74f9ce
I20260806 15:03:36.081707 11991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.072628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:42863 (local address 127.0.236.126:39739)
0806 15:03:36.072668 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:36.072670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:36.072863 (+ 193us) server_negotiation.cc:407] Connection header received
0806 15:03:36.073305 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:36.073307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:36.073320 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:36.073370 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:36.074113 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.074872 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.076332 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.076647 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.076718 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.079176 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:36.079178 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:36.079179 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:36.079187 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:36.080510 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:36.080830 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:36.080831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:36.080831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:36.080878 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:36.081104 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:36.081105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:36.081105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:36.081197 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:36.081251 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:36.081521 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 15:03:36.081614 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:03:36.081830 12072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.072629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39739 (local address 127.0.236.65:42863)
0806 15:03:36.072755 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:36.072787 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:36.073217 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:36.073434 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:36.073438 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:36.073735 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:36.074032 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:36.074038 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.074973 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.074975 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:36.076254 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:36.076258 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.076751 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.076752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:36.076919 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.077468 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:36.077479 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:36.079096 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:36.080574 (+ 1478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:36.080577 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:36.080578 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:36.080774 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:36.080929 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:36.080929 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:36.080929 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:36.081051 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:36.081265 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:36.081272 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:36.081397 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 15:03:36.081487 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 15:03:36.081631 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":69,"threads_started":1}
I20260806 15:03:36.082280 12243 heartbeater.cc:347] Connected to a master server at 127.0.236.126:39739
I20260806 15:03:36.082369 12243 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:36.082547 12243 heartbeater.cc:511] Master 127.0.236.126:39739 requested a full tablet report, sending...
I20260806 15:03:36.084322 11918 ts_manager.cc:194] Registered new tserver with Master: b584a933f35a49678c7f0adde4e4c211 (127.0.236.65:46041)
I20260806 15:03:36.084903 11918 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.236.65@KRBTEST.COM'} at 127.0.236.65:42863
I20260806 15:03:36.088400 12251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.083178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39739 (local address 127.0.236.65:48783)
0806 15:03:36.083304 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:36.083313 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:36.083358 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:36.083465 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:36.083466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:36.083624 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:36.083669 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:36.083670 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.084360 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.084362 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:36.085131 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:36.085134 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.085490 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.085491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:36.085560 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.085933 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:36.085938 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:36.086188 (+ 250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:36.087533 (+ 1345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:36.087534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:36.087535 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:36.087690 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:36.087849 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:36.087850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:36.087850 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:36.087905 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:36.088112 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:36.088113 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:36.088154 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 15:03:36.088214 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 15:03:36.088306 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
I20260806 15:03:36.088393 11991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.083324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:48783 (local address 127.0.236.126:39739)
0806 15:03:36.083374 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:36.083376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:36.083383 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:03:36.083400 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:36.083401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:36.083417 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:36.083463 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:36.083730 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.084265 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.085202 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.085425 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.085481 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.086264 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:36.086265 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:36.086266 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:36.086272 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:36.087464 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:36.087748 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:36.087749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:36.087749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:36.087798 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:36.087961 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:36.087961 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:36.087962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:36.088060 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:36.088113 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:36.088243 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 15:03:36.088305 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
WARNING: no policy specified for oryx/127.0.236.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.236.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.236.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/oryx_127.0.236.66.keytab.
Entry for principal oryx/127.0.236.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/oryx_127.0.236.66.keytab.
I20260806 15:03:36.105453 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:0
--local_ip_for_outbound_sockets=127.0.236.66
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:39739
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/oryx_127.0.236.66.keytab
--principal=oryx/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:42293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:36.182763 12255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:36.182955 12255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:36.182986 12255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:36.184101 12255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:36.184142 12255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:36.184170 12255 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 15:03:36.184192 12255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:03:36.185607 12255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:42293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/oryx_127.0.236.66.keytab
--principal=oryx/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.236.126:39739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:36.185830 12255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:36.186036 12255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:36.190496 12263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:36.190584 12255 server_base.cc:1034] running on GCE node
W20260806 15:03:36.190508 12262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:36.190510 12266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:36.190959 12255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:36.191190 12255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:36.192320 12255 hybrid_clock.cc:648] HybridClock initialized: now 1786028616192312 us; error 26 us; skew 500 ppm
Aug 06 15:03:36 dist-test-slave-tc2s krb5kdc[11848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028616, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:36.194736 12255 init.cc:377] Logged in from keytab as oryx/127.0.236.66@KRBTEST.COM (short username oryx)
I20260806 15:03:36.195137 12255 webserver.cc:460] Webserver started at http://127.0.236.66:32913/ using document root <none> and password file <none>
I20260806 15:03:36.195294 12255 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:36.195330 12255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:36.195421 12255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:36.196306 12255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-1/data/instance:
uuid: "c4eb9691b8c3440c925e003fe1a311f5"
format_stamp: "Formatted at 2026-08-06 15:03:36 on dist-test-slave-tc2s"
server_key: "483abd68ede77eae5f6417ed5fd80cce"
server_key_iv: "f91a2047ad54460b871e31052767af04"
server_key_version: "encryptionkey@0"
I20260806 15:03:36.196583 12255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-1/wal/instance:
uuid: "c4eb9691b8c3440c925e003fe1a311f5"
format_stamp: "Formatted at 2026-08-06 15:03:36 on dist-test-slave-tc2s"
server_key: "483abd68ede77eae5f6417ed5fd80cce"
server_key_iv: "f91a2047ad54460b871e31052767af04"
server_key_version: "encryptionkey@0"
I20260806 15:03:36.197676 12255 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 15:03:36.198324 12275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:36.198462 12255 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:36.198540 12255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "c4eb9691b8c3440c925e003fe1a311f5"
format_stamp: "Formatted at 2026-08-06 15:03:36 on dist-test-slave-tc2s"
server_key: "483abd68ede77eae5f6417ed5fd80cce"
server_key_iv: "f91a2047ad54460b871e31052767af04"
server_key_version: "encryptionkey@0"
I20260806 15:03:36.198597 12255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:36.215312 12255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:36.216826 12255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:36.216959 12255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:36.217413 12255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:36.217453 12255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:36.217479 12255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:36.217495 12255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 15:03:36 dist-test-slave-tc2s krb5kdc[11848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028616, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM
I20260806 15:03:36.224201 12255 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:44243
I20260806 15:03:36.224275 12444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:44243 every 8 connection(s)
I20260806 15:03:36.224526 12255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
I20260806 15:03:36.226212 11991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.217754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:33477 (local address 127.0.236.126:39739)
0806 15:03:36.217819 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:36.217821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:36.218002 (+ 181us) server_negotiation.cc:407] Connection header received
0806 15:03:36.218592 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:36.218593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:36.218602 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:36.218641 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:36.219430 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.219932 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.220908 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.221110 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.221165 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.223793 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:36.223794 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:36.223795 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:36.223801 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:36.225141 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:36.225520 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:36.225521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:36.225521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:36.225571 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:36.225723 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:36.225723 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:36.225723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:36.225812 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:36.225851 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:36.226088 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 15:03:36.226150 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 15:03:36.226327 12289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.217718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39739 (local address 127.0.236.66:33477)
0806 15:03:36.217858 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:36.217927 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:36.218500 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:36.218708 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:36.218712 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:36.219048 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:36.219323 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:36.219329 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.220017 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.220018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:36.220819 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:36.220822 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.221179 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.221180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:36.221303 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.221711 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:36.221727 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:36.223701 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:36.225205 (+ 1504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:36.225208 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:36.225209 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:36.225460 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:36.225609 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:36.225609 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:36.225609 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:36.225690 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:36.225844 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:36.225847 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:36.225989 (+ 142us) client_negotiation.cc:769] Sending connection context
0806 15:03:36.226064 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 15:03:36.226171 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":68,"threads_started":1}
I20260806 15:03:36.226753 12446 heartbeater.cc:347] Connected to a master server at 127.0.236.126:39739
I20260806 15:03:36.226824 12446 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:36.226982 12446 heartbeater.cc:511] Master 127.0.236.126:39739 requested a full tablet report, sending...
I20260806 15:03:36.227344 11916 ts_manager.cc:194] Registered new tserver with Master: c4eb9691b8c3440c925e003fe1a311f5 (127.0.236.66:44243)
I20260806 15:03:36.227820 11916 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.236.66@KRBTEST.COM'} at 127.0.236.66:33477
I20260806 15:03:36.229529 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 12255
I20260806 15:03:36.229638 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-1/wal/instance
I20260806 15:03:36.229833 945 external_mini_cluster.cc:1428] Setting key 62109742c7cd5484754e3dc775f226e4
I20260806 15:03:36.232617 12455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.227648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39739 (local address 127.0.236.66:52277)
0806 15:03:36.227790 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:36.227799 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:36.227857 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:36.227971 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:36.227972 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:36.228163 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:36.228219 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:36.228220 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.228923 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.228924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:36.229427 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:36.229429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.229765 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.229767 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:36.229832 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.230221 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:36.230225 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:36.230480 (+ 255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:36.231718 (+ 1238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:36.231719 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:36.231720 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:36.231918 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:36.232057 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:36.232058 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:36.232058 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:36.232113 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:36.232401 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:36.232402 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:36.232438 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 15:03:36.232485 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 15:03:36.232553 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":66,"threads_started":1}
I20260806 15:03:36.232661 11991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.227714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:52277 (local address 127.0.236.126:39739)
0806 15:03:36.227751 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:36.227753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:36.227846 (+ 93us) server_negotiation.cc:407] Connection header received
0806 15:03:36.227902 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:36.227903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:36.227913 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:36.227962 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:36.228283 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.228834 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.229494 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.229700 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.229748 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.230557 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:36.230558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:36.230558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:36.230565 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:36.231649 (+ 1084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:36.231980 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:36.231981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:36.231981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:36.232021 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:36.232166 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:36.232166 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:36.232167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:36.232341 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:36.232407 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:36.232510 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 15:03:36.232582 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for oryx/127.0.236.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.236.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.236.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/oryx_127.0.236.67.keytab.
Entry for principal oryx/127.0.236.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/oryx_127.0.236.67.keytab.
I20260806 15:03:36.254419 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:0
--local_ip_for_outbound_sockets=127.0.236.67
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:39739
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/oryx_127.0.236.67.keytab
--principal=oryx/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:42293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:36.332412 12458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:36.332608 12458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:36.332638 12458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:36.333734 12458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:36.333770 12458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:36.333797 12458 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 15:03:36.333819 12458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:03:36.335258 12458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:42293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/krb5kdc/oryx_127.0.236.67.keytab
--principal=oryx/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.236.126:39739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:36.335472 12458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:36.335669 12458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:36.340183 12468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:36.340306 12458 server_base.cc:1034] running on GCE node
W20260806 15:03:36.340194 12471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:36.340204 12465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:36.340637 12458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:36.340874 12458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:36.342003 12458 hybrid_clock.cc:648] HybridClock initialized: now 1786028616341986 us; error 28 us; skew 500 ppm
Aug 06 15:03:36 dist-test-slave-tc2s krb5kdc[11848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028616, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:36.346761 12458 init.cc:377] Logged in from keytab as oryx/127.0.236.67@KRBTEST.COM (short username oryx)
I20260806 15:03:36.347164 12458 webserver.cc:460] Webserver started at http://127.0.236.67:43879/ using document root <none> and password file <none>
I20260806 15:03:36.347354 12458 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:36.347399 12458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:36.347522 12458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:36.348482 12458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-2/data/instance:
uuid: "560d593b247846858b20f9d7ea174532"
format_stamp: "Formatted at 2026-08-06 15:03:36 on dist-test-slave-tc2s"
server_key: "cbcac6110e5da6916ee7c1ac077f5e5a"
server_key_iv: "36fcfbb821323a58df96ec28c7bfa3fe"
server_key_version: "encryptionkey@0"
I20260806 15:03:36.348798 12458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-2/wal/instance:
uuid: "560d593b247846858b20f9d7ea174532"
format_stamp: "Formatted at 2026-08-06 15:03:36 on dist-test-slave-tc2s"
server_key: "cbcac6110e5da6916ee7c1ac077f5e5a"
server_key_iv: "36fcfbb821323a58df96ec28c7bfa3fe"
server_key_version: "encryptionkey@0"
I20260806 15:03:36.349960 12458 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:03:36.350615 12480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:36.350747 12458 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:36.350826 12458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "560d593b247846858b20f9d7ea174532"
format_stamp: "Formatted at 2026-08-06 15:03:36 on dist-test-slave-tc2s"
server_key: "cbcac6110e5da6916ee7c1ac077f5e5a"
server_key_iv: "36fcfbb821323a58df96ec28c7bfa3fe"
server_key_version: "encryptionkey@0"
I20260806 15:03:36.350888 12458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:36.367300 12458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:36.369213 12458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:36.369338 12458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:36.369796 12458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:36.369833 12458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:36.369858 12458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:36.369875 12458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 15:03:36 dist-test-slave-tc2s krb5kdc[11848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028616, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM
I20260806 15:03:36.376314 12458 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:45805
I20260806 15:03:36.376452 12646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:45805 every 8 connection(s)
I20260806 15:03:36.376648 12458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
I20260806 15:03:36.378384 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 12458
I20260806 15:03:36.378485 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-2/wal/instance
I20260806 15:03:36.378665 945 external_mini_cluster.cc:1428] Setting key e1e0ec3b24778cbb44cdeb862d557470
I20260806 15:03:36.378723 11991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.370172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:59625 (local address 127.0.236.126:39739)
0806 15:03:36.370227 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:36.370229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:36.370430 (+ 201us) server_negotiation.cc:407] Connection header received
0806 15:03:36.370878 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:36.370879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:36.370893 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:36.370945 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:36.371563 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.372069 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.373027 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.373216 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.373271 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.375972 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:36.375974 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:36.375974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:36.375981 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:36.377388 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:36.377782 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:36.377784 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:36.377784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:36.377841 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:36.378079 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:36.378081 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:36.378081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:36.378195 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:36.378251 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:36.378550 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 15:03:36.378633 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 15:03:36.378875 12495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.370164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39739 (local address 127.0.236.67:59625)
0806 15:03:36.370306 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:36.370354 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:36.370783 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:36.371007 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:36.371011 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:36.371286 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:36.371473 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:36.371477 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.372151 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.372152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:36.372942 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:36.372944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.373284 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.373285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:36.373402 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.373822 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:36.373831 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:36.375882 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:36.377461 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:36.377468 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:36.377470 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:36.377713 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:36.377913 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:36.377913 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:36.377914 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:36.378022 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:36.378268 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:36.378274 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:36.378411 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 15:03:36.378508 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 15:03:36.378673 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":70,"threads_started":1}
I20260806 15:03:36.379273 12647 heartbeater.cc:347] Connected to a master server at 127.0.236.126:39739
I20260806 15:03:36.379352 12647 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:36.379508 12647 heartbeater.cc:511] Master 127.0.236.126:39739 requested a full tablet report, sending...
I20260806 15:03:36.379817 11916 ts_manager.cc:194] Registered new tserver with Master: 560d593b247846858b20f9d7ea174532 (127.0.236.67:45805)
I20260806 15:03:36.380177 11916 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.236.67@KRBTEST.COM'} at 127.0.236.67:59625
I20260806 15:03:36.384965 11991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.380263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:52397 (local address 127.0.236.126:39739)
0806 15:03:36.380307 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:36.380309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:36.380469 (+ 160us) server_negotiation.cc:407] Connection header received
0806 15:03:36.380497 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:36.380498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:36.380509 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:36.380578 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:36.380905 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.381340 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.381988 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.382217 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.382278 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.382944 (+ 666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:36.382945 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:36.382946 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:36.382951 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:36.384137 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:36.384440 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:36.384440 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:36.384441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:36.384486 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:36.384621 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:36.384622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:36.384622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:36.384706 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:36.384748 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:36.384852 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 15:03:36.384909 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 15:03:36.384972 12655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.380203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39739 (local address 127.0.236.67:52397)
0806 15:03:36.380379 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:36.380389 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:36.380436 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:36.380585 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:36.380585 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:36.380768 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:36.380826 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:36.380828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.381413 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.381413 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:36.381916 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:36.381918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.382280 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.382280 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:36.382328 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.382634 (+ 306us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:36.382638 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:36.382874 (+ 236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:36.384203 (+ 1329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:36.384204 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:36.384205 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:36.384382 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:36.384531 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:36.384531 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:36.384532 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:36.384580 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:36.384745 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:36.384745 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:36.384778 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 15:03:36.384836 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 15:03:36.384906 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":75,"threads_started":1}
I20260806 15:03:36.390405 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 15:03:36 dist-test-slave-tc2s krb5kdc[11848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028616, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 15:03:36 dist-test-slave-tc2s krb5kdc[11848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:36 dist-test-slave-tc2s krb5kdc[11848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 15:03:36.410575 11991 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:36.405930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38966 (local address 127.0.236.126:39739)
0806 15:03:36.405971 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:36.405973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:36.405982 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:03:36.406021 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:36.406022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:36.406031 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:36.406078 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:36.406569 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.407031 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.407736 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.407916 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.407969 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.410503 (+ 2534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38966: BlockingRecv error: recv got EOF from 127.0.0.1:38966 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 06 15:03:36 dist-test-slave-tc2s krb5kdc[11848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028616, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM
I20260806 15:03:36.417049 11991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.411256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38970 (local address 127.0.236.126:39739)
0806 15:03:36.411310 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:36.411311 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:36.411358 (+ 47us) server_negotiation.cc:407] Connection header received
0806 15:03:36.411416 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:36.411416 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:36.411425 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:36.411469 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:36.411869 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.412312 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.412947 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.413131 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.413181 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.414986 (+ 1805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:36.414987 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:36.414988 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:36.414994 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:36.416131 (+ 1137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:36.416415 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:36.416416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:36.416417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:36.416467 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:36.416624 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:36.416625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:36.416625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:36.416722 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:36.416804 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:36.416897 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 15:03:36.416969 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 15:03:36.418373 11916 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38970:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:03:36.419808 11916 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:03:36.425560 12682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.421864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:45805 (local address 127.0.0.1:55312)
0806 15:03:36.422091 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:36.422101 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:36.422149 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:36.422389 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:36.422389 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:36.422391 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:36.422419 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:36.422420 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.423628 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.423629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:36.424653 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:36.424655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.425269 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.425270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:36.425352 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.425366 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 15:03:36.425437 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 15:03:36.425493 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":63,"threads_started":1}
I20260806 15:03:36.425572 12684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.422028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55312 (local address 127.0.236.67:45805)
0806 15:03:36.422239 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:36.422243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:36.422252 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:03:36.422284 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:36.422285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:36.422300 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:36.422377 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:36.422480 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.423546 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.424721 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.425192 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.425253 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.425325 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:36.425456 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 15:03:36.425489 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":69,"threads_started":1}
I20260806 15:03:36.425765 12678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.421619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:44243 (local address 127.0.0.1:52844)
0806 15:03:36.422037 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:36.422046 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:36.422093 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:36.422331 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:36.422332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:36.422334 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:36.422390 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:36.422391 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.423322 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.423323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:36.424545 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:36.424548 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.425550 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.425551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:36.425612 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.425616 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:03:36.425657 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 15:03:36.425694 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":59,"threads_started":1}
I20260806 15:03:36.426155 12680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.421725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52844 (local address 127.0.236.66:44243)
0806 15:03:36.421929 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:36.421934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:36.422147 (+ 213us) server_negotiation.cc:407] Connection header received
0806 15:03:36.422186 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:36.422188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:36.422202 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:36.422267 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:36.422478 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.423244 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.424640 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.425387 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.425452 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.425515 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:36.426024 (+ 509us) server_negotiation.cc:299] Negotiation successful
0806 15:03:36.426064 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
I20260806 15:03:36.426213 12553 tablet_service.cc:1467] Processing CreateTablet for tablet 950f7cddafa94e32b8e85701c0cdb91e (DEFAULT_TABLE table=test-table [id=3b81928e69fd4928a818ce12c9146ecf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:36.426574 12553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 950f7cddafa94e32b8e85701c0cdb91e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:36.426710 12344 tablet_service.cc:1467] Processing CreateTablet for tablet 950f7cddafa94e32b8e85701c0cdb91e (DEFAULT_TABLE table=test-table [id=3b81928e69fd4928a818ce12c9146ecf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:36.427028 12679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.421611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:46041 (local address 127.0.0.1:39000)
0806 15:03:36.421932 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:36.421981 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:36.422053 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:36.422282 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:36.422285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:36.422287 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:36.422346 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:36.422348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.424732 (+ 2384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.424732 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:36.425754 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:36.425756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.426802 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.426804 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:36.426877 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.426880 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:03:36.426929 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 15:03:36.426966 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":52,"threads_started":1}
I20260806 15:03:36.427057 12344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 950f7cddafa94e32b8e85701c0cdb91e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:36.427263 12681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.421785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39000 (local address 127.0.236.65:46041)
0806 15:03:36.421995 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:36.422002 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:36.422132 (+ 130us) server_negotiation.cc:407] Connection header received
0806 15:03:36.422174 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:36.422176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:36.422191 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:36.422266 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:36.422438 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.423549 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.425841 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.426629 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.426693 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.426766 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:36.427131 (+ 365us) server_negotiation.cc:299] Negotiation successful
0806 15:03:36.427174 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":69,"threads_started":1}
I20260806 15:03:36.427887 12134 tablet_service.cc:1467] Processing CreateTablet for tablet 950f7cddafa94e32b8e85701c0cdb91e (DEFAULT_TABLE table=test-table [id=3b81928e69fd4928a818ce12c9146ecf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:36.428205 12134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 950f7cddafa94e32b8e85701c0cdb91e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:36.430960 12685 tablet_bootstrap.cc:492] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532: Bootstrap starting.
I20260806 15:03:36.431573 12685 tablet_bootstrap.cc:654] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:36.431722 12687 tablet_bootstrap.cc:492] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5: Bootstrap starting.
I20260806 15:03:36.431941 12685 log.cc:824] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:36.432324 12687 tablet_bootstrap.cc:654] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:36.432586 12687 log.cc:824] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:36.432682 12685 tablet_bootstrap.cc:492] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532: No bootstrap required, opened a new log
I20260806 15:03:36.432761 12685 ts_tablet_manager.cc:1397] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:03:36.433259 12687 tablet_bootstrap.cc:492] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5: No bootstrap required, opened a new log
I20260806 15:03:36.433321 12687 ts_tablet_manager.cc:1397] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:03:36.434655 12687 raft_consensus.cc:348] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:36.434782 12687 raft_consensus.cc:374] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:36.434815 12687 raft_consensus.cc:729] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4eb9691b8c3440c925e003fe1a311f5, State: Initialized, Role: FOLLOWER
I20260806 15:03:36.434734 12685 raft_consensus.cc:348] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:36.434885 12685 raft_consensus.cc:374] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:36.434930 12685 raft_consensus.cc:729] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 560d593b247846858b20f9d7ea174532, State: Initialized, Role: FOLLOWER
I20260806 15:03:36.434932 12687 consensus_queue.cc:260] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:36.435053 12685 consensus_queue.cc:260] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:36.435201 12687 ts_tablet_manager.cc:1428] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:03:36.435254 12446 heartbeater.cc:503] Master 127.0.236.126:39739 was elected leader, sending a full tablet report...
I20260806 15:03:36.435309 12685 ts_tablet_manager.cc:1428] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 15:03:36.439365 12647 heartbeater.cc:503] Master 127.0.236.126:39739 was elected leader, sending a full tablet report...
I20260806 15:03:36.439689 12691 tablet_bootstrap.cc:492] T 950f7cddafa94e32b8e85701c0cdb91e P b584a933f35a49678c7f0adde4e4c211: Bootstrap starting.
I20260806 15:03:36.440191 12691 tablet_bootstrap.cc:654] T 950f7cddafa94e32b8e85701c0cdb91e P b584a933f35a49678c7f0adde4e4c211: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:36.440426 12691 log.cc:824] T 950f7cddafa94e32b8e85701c0cdb91e P b584a933f35a49678c7f0adde4e4c211: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:36.441239 12691 tablet_bootstrap.cc:492] T 950f7cddafa94e32b8e85701c0cdb91e P b584a933f35a49678c7f0adde4e4c211: No bootstrap required, opened a new log
I20260806 15:03:36.441325 12691 ts_tablet_manager.cc:1397] T 950f7cddafa94e32b8e85701c0cdb91e P b584a933f35a49678c7f0adde4e4c211: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:03:36.443485 12691 raft_consensus.cc:348] T 950f7cddafa94e32b8e85701c0cdb91e P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:36.443625 12691 raft_consensus.cc:374] T 950f7cddafa94e32b8e85701c0cdb91e P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:36.443650 12691 raft_consensus.cc:729] T 950f7cddafa94e32b8e85701c0cdb91e P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b584a933f35a49678c7f0adde4e4c211, State: Initialized, Role: FOLLOWER
I20260806 15:03:36.443753 12691 consensus_queue.cc:260] T 950f7cddafa94e32b8e85701c0cdb91e P b584a933f35a49678c7f0adde4e4c211 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:36.443979 12691 ts_tablet_manager.cc:1428] T 950f7cddafa94e32b8e85701c0cdb91e P b584a933f35a49678c7f0adde4e4c211: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 15:03:36.444011 12243 heartbeater.cc:503] Master 127.0.236.126:39739 was elected leader, sending a full tablet report...
W20260806 15:03:36.475006 12447 tablet.cc:2369] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 15:03:36.579695 12244 tablet.cc:2369] T 950f7cddafa94e32b8e85701c0cdb91e P b584a933f35a49678c7f0adde4e4c211: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 15:03:36.627749 12648 tablet.cc:2369] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:03:36.717797 12693 raft_consensus.cc:482] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:03:36.717953 12693 raft_consensus.cc:504] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:36.718309 12693 leader_election.cc:288] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b584a933f35a49678c7f0adde4e4c211 (127.0.236.65:46041), c4eb9691b8c3440c925e003fe1a311f5 (127.0.236.66:44243)
I20260806 15:03:36.720024 12692 raft_consensus.cc:482] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:03:36.720132 12692 raft_consensus.cc:504] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:36.720418 12692 leader_election.cc:288] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b584a933f35a49678c7f0adde4e4c211 (127.0.236.65:46041), 560d593b247846858b20f9d7ea174532 (127.0.236.67:45805)
I20260806 15:03:36.721372 12495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.718505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:46041 (local address 127.0.236.67:48463)
0806 15:03:36.718560 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:36.718571 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:36.718613 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:36.718788 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:36.718788 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:36.718790 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:36.718861 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:36.718863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.719515 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.719515 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:36.720369 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:36.720371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.721110 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.721111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:36.721204 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.721208 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:03:36.721258 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 15:03:36.721302 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 15:03:36.721390 12681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.718543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:48463 (local address 127.0.236.65:46041)
0806 15:03:36.718660 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:36.718662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:36.718671 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:03:36.718699 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:36.718700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:36.718709 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:36.718776 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:36.718916 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.719446 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.720462 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.721020 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.721098 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.721108 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:36.721268 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 15:03:36.721322 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 15:03:36.721832 12165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "950f7cddafa94e32b8e85701c0cdb91e" candidate_uuid: "560d593b247846858b20f9d7ea174532" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b584a933f35a49678c7f0adde4e4c211" is_pre_election: true
I20260806 15:03:36.722004 12165 raft_consensus.cc:2454] T 950f7cddafa94e32b8e85701c0cdb91e P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 560d593b247846858b20f9d7ea174532 in term 0.
I20260806 15:03:36.722208 12482 leader_election.cc:302] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 560d593b247846858b20f9d7ea174532, b584a933f35a49678c7f0adde4e4c211; no voters:
I20260806 15:03:36.722327 12693 raft_consensus.cc:2781] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:03:36.722393 12693 raft_consensus.cc:482] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:03:36.722395 12680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.718599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:44131 (local address 127.0.236.66:44243)
0806 15:03:36.718648 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:36.718650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:36.718748 (+ 98us) server_negotiation.cc:407] Connection header received
0806 15:03:36.718777 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:36.718778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:36.718786 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:36.718836 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:36.719051 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.719508 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.720593 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.721147 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.721204 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.721212 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:36.722300 (+ 1088us) server_negotiation.cc:299] Negotiation successful
0806 15:03:36.722342 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 15:03:36.722429 12693 raft_consensus.cc:3037] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:36.722514 12696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.718505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:44243 (local address 127.0.236.67:44131)
0806 15:03:36.718688 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:36.718696 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:36.718740 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:36.718883 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:36.718884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:36.718886 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:36.718974 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:36.718976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.719588 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.719589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:36.720499 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:36.720502 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.721220 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.722159 (+ 939us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:36.722233 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.722236 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:03:36.722276 (+ 40us) client_negotiation.cc:240] Negotiation successful
0806 15:03:36.722313 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":13,"thread_start_us":98,"threads_started":1}
I20260806 15:03:36.722805 12376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "950f7cddafa94e32b8e85701c0cdb91e" candidate_uuid: "560d593b247846858b20f9d7ea174532" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4eb9691b8c3440c925e003fe1a311f5" is_pre_election: true
I20260806 15:03:36.722877 12376 raft_consensus.cc:2454] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 560d593b247846858b20f9d7ea174532 in term 0.
I20260806 15:03:36.723645 12693 raft_consensus.cc:504] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:36.723762 12693 leader_election.cc:288] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532 [CANDIDATE]: Term 1 election: Requested vote from peers b584a933f35a49678c7f0adde4e4c211 (127.0.236.65:46041), c4eb9691b8c3440c925e003fe1a311f5 (127.0.236.66:44243)
I20260806 15:03:36.723918 12376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "950f7cddafa94e32b8e85701c0cdb91e" candidate_uuid: "560d593b247846858b20f9d7ea174532" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4eb9691b8c3440c925e003fe1a311f5"
I20260806 15:03:36.723995 12376 raft_consensus.cc:3037] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:36.724200 12165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "950f7cddafa94e32b8e85701c0cdb91e" candidate_uuid: "560d593b247846858b20f9d7ea174532" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b584a933f35a49678c7f0adde4e4c211"
I20260806 15:03:36.724282 12165 raft_consensus.cc:3037] T 950f7cddafa94e32b8e85701c0cdb91e P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:36.724331 12684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.721206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:38609 (local address 127.0.236.67:45805)
0806 15:03:36.721255 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:36.721256 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:36.721262 (+ 6us) server_negotiation.cc:407] Connection header received
0806 15:03:36.721276 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:36.721277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:36.721285 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:36.721329 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:36.721510 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.722084 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.723547 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.723966 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.724028 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.724035 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:36.724249 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 15:03:36.724280 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 15:03:36.724519 12698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.720697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:45805 (local address 127.0.236.66:38609)
0806 15:03:36.721052 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:36.721067 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:36.721119 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:36.721369 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:36.721370 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:36.721372 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:36.721441 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:36.721443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.722381 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.722382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:36.723473 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:36.723476 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.724067 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.724068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:36.724164 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.724168 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:03:36.724395 (+ 227us) client_negotiation.cc:240] Negotiation successful
0806 15:03:36.724435 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":52,"threads_started":1}
I20260806 15:03:36.724784 12376 raft_consensus.cc:2454] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 560d593b247846858b20f9d7ea174532 in term 1.
I20260806 15:03:36.724939 12484 leader_election.cc:302] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 560d593b247846858b20f9d7ea174532, c4eb9691b8c3440c925e003fe1a311f5; no voters:
I20260806 15:03:36.725016 12693 raft_consensus.cc:2781] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:36.725112 12165 raft_consensus.cc:2454] T 950f7cddafa94e32b8e85701c0cdb91e P b584a933f35a49678c7f0adde4e4c211 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 560d593b247846858b20f9d7ea174532 in term 1.
I20260806 15:03:36.725124 12693 raft_consensus.cc:686] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532 [term 1 LEADER]: Becoming Leader. State: Replica: 560d593b247846858b20f9d7ea174532, State: Running, Role: LEADER
I20260806 15:03:36.725203 12693 consensus_queue.cc:237] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:36.725510 12580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "950f7cddafa94e32b8e85701c0cdb91e" candidate_uuid: "c4eb9691b8c3440c925e003fe1a311f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "560d593b247846858b20f9d7ea174532" is_pre_election: true
I20260806 15:03:36.726069 11916 catalog_manager.cc:5495] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532 reported cstate change: term changed from 0 to 1, leader changed from <none> to 560d593b247846858b20f9d7ea174532 (127.0.236.67). New cstate: current_term: 1 leader_uuid: "560d593b247846858b20f9d7ea174532" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } health_report { overall_health: HEALTHY } } }
I20260806 15:03:36.726392 12289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.720609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:46041 (local address 127.0.236.66:60625)
0806 15:03:36.720783 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:36.720792 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:36.720821 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:36.721396 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:36.721397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:36.721399 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:36.721468 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:36.721470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.723969 (+ 2499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.723969 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:36.725400 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:36.725402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.726170 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.726171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:36.726242 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.726246 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:03:36.726277 (+ 31us) client_negotiation.cc:240] Negotiation successful
0806 15:03:36.726320 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159}
I20260806 15:03:36.726629 12699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.720938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:60625 (local address 127.0.236.65:46041)
0806 15:03:36.721273 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:36.721276 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:36.721282 (+ 6us) server_negotiation.cc:407] Connection header received
0806 15:03:36.721303 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:36.721303 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:36.721312 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:36.721538 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:36.721587 (+ 49us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.722232 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.725524 (+ 3292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.726071 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.726506 (+ 435us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.726517 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:36.726536 (+ 19us) server_negotiation.cc:299] Negotiation successful
0806 15:03:36.726574 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":61,"threads_started":1}
I20260806 15:03:36.726799 12165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "950f7cddafa94e32b8e85701c0cdb91e" candidate_uuid: "c4eb9691b8c3440c925e003fe1a311f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b584a933f35a49678c7f0adde4e4c211" is_pre_election: true
I20260806 15:03:36.726886 12165 raft_consensus.cc:2379] T 950f7cddafa94e32b8e85701c0cdb91e P b584a933f35a49678c7f0adde4e4c211 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c4eb9691b8c3440c925e003fe1a311f5 in current term 1: Already voted for candidate 560d593b247846858b20f9d7ea174532 in this term.
I20260806 15:03:36.727080 12278 leader_election.cc:302] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4eb9691b8c3440c925e003fe1a311f5; no voters: 560d593b247846858b20f9d7ea174532, b584a933f35a49678c7f0adde4e4c211
I20260806 15:03:36.727203 12692 raft_consensus.cc:2726] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 15:03:36.776849 12684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:36.774642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55328 (local address 127.0.236.67:45805)
0806 15:03:36.774689 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:36.774691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:36.774699 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:36.774776 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:36.774777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:36.774797 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:36.774853 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:03:36.774978 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.775423 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.776154 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:36.776334 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:36.776387 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:36.776560 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:03:36.776657 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:03:36.776723 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:36.776761 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 15:03:36.776779 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 15:03:36.778520 12165 raft_consensus.cc:1261] T 950f7cddafa94e32b8e85701c0cdb91e P b584a933f35a49678c7f0adde4e4c211 [term 1 FOLLOWER]: Refusing update from remote peer 560d593b247846858b20f9d7ea174532: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:36.778533 12376 raft_consensus.cc:1261] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5 [term 1 FOLLOWER]: Refusing update from remote peer 560d593b247846858b20f9d7ea174532: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:36.778798 12700 consensus_queue.cc:1035] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:36.778914 12693 consensus_queue.cc:1035] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:36.780992 12706 mvcc.cc:205] Tried to move back new op lower bound from 7315573214322413568 to 7315573214106906624. Current Snapshot: MvccSnapshot[applied={T|T < 7315573214322413568}]
I20260806 15:03:36.781400 12709 mvcc.cc:205] Tried to move back new op lower bound from 7315573214322413568 to 7315573214106906624. Current Snapshot: MvccSnapshot[applied={T|T < 7315573214322413568 or (T in {7315573214322413568})}]
I20260806 15:03:36.782020 12711 mvcc.cc:205] Tried to move back new op lower bound from 7315573214322413568 to 7315573214106906624. Current Snapshot: MvccSnapshot[applied={T|T < 7315573214322413568 or (T in {7315573214322413568})}]
I20260806 15:03:36.784418 11918 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:36.784504 11918 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:36.785177 11918 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef: Sending DeleteTablet for 3 replicas of tablet 950f7cddafa94e32b8e85701c0cdb91e
I20260806 15:03:36.785485 12134 tablet_service.cc:1514] Processing DeleteTablet for tablet 950f7cddafa94e32b8e85701c0cdb91e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:36 UTC) from {username='oryx', principal='oryx/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:39000
I20260806 15:03:36.785507 12553 tablet_service.cc:1514] Processing DeleteTablet for tablet 950f7cddafa94e32b8e85701c0cdb91e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:36 UTC) from {username='oryx', principal='oryx/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:55312
I20260806 15:03:36.785499 12344 tablet_service.cc:1514] Processing DeleteTablet for tablet 950f7cddafa94e32b8e85701c0cdb91e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:36 UTC) from {username='oryx', principal='oryx/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:52844
I20260806 15:03:36.785707 12728 tablet_replica.cc:330] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5: stopping tablet replica
I20260806 15:03:36.785806 12728 raft_consensus.cc:2229] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:36.785831 12734 tablet_replica.cc:330] T 950f7cddafa94e32b8e85701c0cdb91e P b584a933f35a49678c7f0adde4e4c211: stopping tablet replica
I20260806 15:03:36.785934 12728 raft_consensus.cc:2258] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:36.785930 11918 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38970:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:03:36.786079 11918 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:03:36.786298 12734 raft_consensus.cc:2229] T 950f7cddafa94e32b8e85701c0cdb91e P b584a933f35a49678c7f0adde4e4c211 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:36.786365 12728 ts_tablet_manager.cc:1905] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:36.786520 12734 raft_consensus.cc:2258] T 950f7cddafa94e32b8e85701c0cdb91e P b584a933f35a49678c7f0adde4e4c211 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:36.787199 12134 tablet_service.cc:1467] Processing CreateTablet for tablet e572cb484ed34b08b116bd534571e61c (DEFAULT_TABLE table=test-table [id=406e61bb3a78400699d5be8f2d6bea3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:36.787228 12553 tablet_service.cc:1467] Processing CreateTablet for tablet e572cb484ed34b08b116bd534571e61c (DEFAULT_TABLE table=test-table [id=406e61bb3a78400699d5be8f2d6bea3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:36.787317 12134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e572cb484ed34b08b116bd534571e61c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:36.787330 12553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e572cb484ed34b08b116bd534571e61c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:36.787941 12728 ts_tablet_manager.cc:1918] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:03:36.788005 12728 log.cc:1196] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-1/wal/wals/950f7cddafa94e32b8e85701c0cdb91e
I20260806 15:03:36.788255 12728 ts_tablet_manager.cc:1939] T 950f7cddafa94e32b8e85701c0cdb91e P c4eb9691b8c3440c925e003fe1a311f5: Deleting consensus metadata
I20260806 15:03:36.788625 12685 tablet_bootstrap.cc:492] T e572cb484ed34b08b116bd534571e61c P 560d593b247846858b20f9d7ea174532: Bootstrap starting.
I20260806 15:03:36.788770 12691 tablet_bootstrap.cc:492] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211: Bootstrap starting.
I20260806 15:03:36.788842 11899 catalog_manager.cc:4841] TS c4eb9691b8c3440c925e003fe1a311f5 (127.0.236.66:44243): tablet 950f7cddafa94e32b8e85701c0cdb91e (table test-table [id=3b81928e69fd4928a818ce12c9146ecf]) successfully deleted
I20260806 15:03:36.789012 12344 tablet_service.cc:1467] Processing CreateTablet for tablet e572cb484ed34b08b116bd534571e61c (DEFAULT_TABLE table=test-table [id=406e61bb3a78400699d5be8f2d6bea3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:36.789031 12685 tablet_bootstrap.cc:654] T e572cb484ed34b08b116bd534571e61c P 560d593b247846858b20f9d7ea174532: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:36.789103 12344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e572cb484ed34b08b116bd534571e61c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:36.789146 12691 tablet_bootstrap.cc:654] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:36.789615 12685 tablet_bootstrap.cc:492] T e572cb484ed34b08b116bd534571e61c P 560d593b247846858b20f9d7ea174532: No bootstrap required, opened a new log
I20260806 15:03:36.789664 12685 ts_tablet_manager.cc:1397] T e572cb484ed34b08b116bd534571e61c P 560d593b247846858b20f9d7ea174532: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:36.789796 12685 raft_consensus.cc:348] T e572cb484ed34b08b116bd534571e61c P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:36.789867 12685 raft_consensus.cc:374] T e572cb484ed34b08b116bd534571e61c P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:36.789911 12685 raft_consensus.cc:729] T e572cb484ed34b08b116bd534571e61c P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 560d593b247846858b20f9d7ea174532, State: Initialized, Role: FOLLOWER
I20260806 15:03:36.789959 12685 consensus_queue.cc:260] T e572cb484ed34b08b116bd534571e61c P 560d593b247846858b20f9d7ea174532 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:36.790050 12685 ts_tablet_manager.cc:1428] T e572cb484ed34b08b116bd534571e61c P 560d593b247846858b20f9d7ea174532: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:36.790320 12691 tablet_bootstrap.cc:492] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211: No bootstrap required, opened a new log
I20260806 15:03:36.790365 12691 ts_tablet_manager.cc:1397] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:03:36.790478 12687 tablet_bootstrap.cc:492] T e572cb484ed34b08b116bd534571e61c P c4eb9691b8c3440c925e003fe1a311f5: Bootstrap starting.
I20260806 15:03:36.790685 12691 raft_consensus.cc:348] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:36.790747 12691 raft_consensus.cc:374] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:36.790767 12691 raft_consensus.cc:729] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b584a933f35a49678c7f0adde4e4c211, State: Initialized, Role: FOLLOWER
I20260806 15:03:36.790813 12691 consensus_queue.cc:260] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:36.790892 12687 tablet_bootstrap.cc:654] T e572cb484ed34b08b116bd534571e61c P c4eb9691b8c3440c925e003fe1a311f5: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:36.790907 12691 ts_tablet_manager.cc:1428] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:36.791540 12734 ts_tablet_manager.cc:1905] T 950f7cddafa94e32b8e85701c0cdb91e P b584a933f35a49678c7f0adde4e4c211: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:36.791576 12687 tablet_bootstrap.cc:492] T e572cb484ed34b08b116bd534571e61c P c4eb9691b8c3440c925e003fe1a311f5: No bootstrap required, opened a new log
I20260806 15:03:36.791610 12687 ts_tablet_manager.cc:1397] T e572cb484ed34b08b116bd534571e61c P c4eb9691b8c3440c925e003fe1a311f5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:36.791749 12687 raft_consensus.cc:348] T e572cb484ed34b08b116bd534571e61c P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:36.791815 12687 raft_consensus.cc:374] T e572cb484ed34b08b116bd534571e61c P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:36.791843 12687 raft_consensus.cc:729] T e572cb484ed34b08b116bd534571e61c P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4eb9691b8c3440c925e003fe1a311f5, State: Initialized, Role: FOLLOWER
I20260806 15:03:36.791879 12687 consensus_queue.cc:260] T e572cb484ed34b08b116bd534571e61c P c4eb9691b8c3440c925e003fe1a311f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:36.791960 12687 ts_tablet_manager.cc:1428] T e572cb484ed34b08b116bd534571e61c P c4eb9691b8c3440c925e003fe1a311f5: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:36.792889 12734 ts_tablet_manager.cc:1918] T 950f7cddafa94e32b8e85701c0cdb91e P b584a933f35a49678c7f0adde4e4c211: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:03:36.792954 12734 log.cc:1196] T 950f7cddafa94e32b8e85701c0cdb91e P b584a933f35a49678c7f0adde4e4c211: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-0/wal/wals/950f7cddafa94e32b8e85701c0cdb91e
I20260806 15:03:36.793118 12734 ts_tablet_manager.cc:1939] T 950f7cddafa94e32b8e85701c0cdb91e P b584a933f35a49678c7f0adde4e4c211: Deleting consensus metadata
I20260806 15:03:36.793442 11895 catalog_manager.cc:4841] TS b584a933f35a49678c7f0adde4e4c211 (127.0.236.65:46041): tablet 950f7cddafa94e32b8e85701c0cdb91e (table test-table [id=3b81928e69fd4928a818ce12c9146ecf]) successfully deleted
I20260806 15:03:36.794008 12729 tablet_replica.cc:330] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532: stopping tablet replica
I20260806 15:03:36.794090 12729 raft_consensus.cc:2229] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532 [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:03:36.794198 12729 raft_consensus.cc:2258] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:36.794538 12729 ts_tablet_manager.cc:1905] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:36.795625 12729 ts_tablet_manager.cc:1918] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:03:36.795686 12729 log.cc:1196] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/950f7cddafa94e32b8e85701c0cdb91e
I20260806 15:03:36.795899 12729 ts_tablet_manager.cc:1939] T 950f7cddafa94e32b8e85701c0cdb91e P 560d593b247846858b20f9d7ea174532: Deleting consensus metadata
I20260806 15:03:36.796236 11895 catalog_manager.cc:4841] TS 560d593b247846858b20f9d7ea174532 (127.0.236.67:45805): tablet 950f7cddafa94e32b8e85701c0cdb91e (table test-table [id=3b81928e69fd4928a818ce12c9146ecf]) successfully deleted
I20260806 15:03:37.009383 12695 raft_consensus.cc:482] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:03:37.009479 12695 raft_consensus.cc:504] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.009768 12695 leader_election.cc:288] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4eb9691b8c3440c925e003fe1a311f5 (127.0.236.66:44243), 560d593b247846858b20f9d7ea174532 (127.0.236.67:45805)
I20260806 15:03:37.012629 12680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:37.010008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:34685 (local address 127.0.236.66:44243)
0806 15:03:37.010084 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:37.010086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:37.010212 (+ 126us) server_negotiation.cc:407] Connection header received
0806 15:03:37.010269 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:37.010270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:37.010283 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:37.010330 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:37.010464 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.010968 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.011985 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.012338 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.012399 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:37.012416 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:37.012532 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 15:03:37.012567 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 15:03:37.012670 12684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:37.010008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:60021 (local address 127.0.236.67:45805)
0806 15:03:37.010056 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:37.010059 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:37.010225 (+ 166us) server_negotiation.cc:407] Connection header received
0806 15:03:37.010286 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:37.010287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:37.010296 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:37.010341 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:37.010485 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.010956 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.011995 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.012342 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.012401 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:37.012410 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:37.012570 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 15:03:37.012603 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 15:03:37.012683 12741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:37.009961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:44243 (local address 127.0.236.65:34685)
0806 15:03:37.010145 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:37.010154 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:37.010215 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:37.010336 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:37.010337 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:37.010339 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:37.010408 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:37.010410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.011039 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.011040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:37.011904 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:37.011907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.012405 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.012406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:37.012474 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:37.012478 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:03:37.012522 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 15:03:37.012559 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":80,"threads_started":1}
I20260806 15:03:37.012868 12740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:37.010005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:45805 (local address 127.0.236.65:60021)
0806 15:03:37.010155 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:37.010165 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:37.010224 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:37.010348 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:37.010348 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:37.010350 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:37.010418 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:37.010419 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.011028 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.011029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:37.011903 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:37.011906 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.012429 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.012431 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:37.012504 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:37.012508 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:03:37.012552 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 15:03:37.012593 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":23,"thread_start_us":91,"threads_started":1}
I20260806 15:03:37.012965 12376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e572cb484ed34b08b116bd534571e61c" candidate_uuid: "b584a933f35a49678c7f0adde4e4c211" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4eb9691b8c3440c925e003fe1a311f5" is_pre_election: true
I20260806 15:03:37.013055 12376 raft_consensus.cc:2454] T e572cb484ed34b08b116bd534571e61c P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b584a933f35a49678c7f0adde4e4c211 in term 0.
I20260806 15:03:37.013104 12580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e572cb484ed34b08b116bd534571e61c" candidate_uuid: "b584a933f35a49678c7f0adde4e4c211" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "560d593b247846858b20f9d7ea174532" is_pre_election: true
I20260806 15:03:37.013203 12580 raft_consensus.cc:2454] T e572cb484ed34b08b116bd534571e61c P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b584a933f35a49678c7f0adde4e4c211 in term 0.
I20260806 15:03:37.013231 12064 leader_election.cc:302] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b584a933f35a49678c7f0adde4e4c211, c4eb9691b8c3440c925e003fe1a311f5; no voters:
I20260806 15:03:37.013329 12695 raft_consensus.cc:2781] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:03:37.013362 12695 raft_consensus.cc:482] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:03:37.013381 12695 raft_consensus.cc:3037] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:37.014045 12695 raft_consensus.cc:504] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.014169 12695 leader_election.cc:288] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211 [CANDIDATE]: Term 1 election: Requested vote from peers c4eb9691b8c3440c925e003fe1a311f5 (127.0.236.66:44243), 560d593b247846858b20f9d7ea174532 (127.0.236.67:45805)
I20260806 15:03:37.014298 12580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e572cb484ed34b08b116bd534571e61c" candidate_uuid: "b584a933f35a49678c7f0adde4e4c211" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "560d593b247846858b20f9d7ea174532"
I20260806 15:03:37.014358 12580 raft_consensus.cc:3037] T e572cb484ed34b08b116bd534571e61c P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:37.014964 12580 raft_consensus.cc:2454] T e572cb484ed34b08b116bd534571e61c P 560d593b247846858b20f9d7ea174532 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b584a933f35a49678c7f0adde4e4c211 in term 1.
I20260806 15:03:37.015076 12376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e572cb484ed34b08b116bd534571e61c" candidate_uuid: "b584a933f35a49678c7f0adde4e4c211" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4eb9691b8c3440c925e003fe1a311f5"
I20260806 15:03:37.015151 12376 raft_consensus.cc:3037] T e572cb484ed34b08b116bd534571e61c P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:37.015180 12057 leader_election.cc:302] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 560d593b247846858b20f9d7ea174532, b584a933f35a49678c7f0adde4e4c211; no voters:
I20260806 15:03:37.015760 12376 raft_consensus.cc:2454] T e572cb484ed34b08b116bd534571e61c P c4eb9691b8c3440c925e003fe1a311f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b584a933f35a49678c7f0adde4e4c211 in term 1.
I20260806 15:03:37.015854 12695 raft_consensus.cc:2781] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:37.015996 12695 raft_consensus.cc:686] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211 [term 1 LEADER]: Becoming Leader. State: Replica: b584a933f35a49678c7f0adde4e4c211, State: Running, Role: LEADER
I20260806 15:03:37.016060 12695 consensus_queue.cc:237] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.016691 11918 catalog_manager.cc:5495] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211 reported cstate change: term changed from 0 to 1, leader changed from <none> to b584a933f35a49678c7f0adde4e4c211 (127.0.236.65). New cstate: current_term: 1 leader_uuid: "b584a933f35a49678c7f0adde4e4c211" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } health_report { overall_health: UNKNOWN } } }
Aug 06 15:03:37 dist-test-slave-tc2s krb5kdc[11848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028615, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.126@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM
I20260806 15:03:37.100697 12753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:37.094415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39739 (local address 127.0.0.1:38974)
0806 15:03:37.094572 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:37.094582 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:37.094626 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:37.094735 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:37.094736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:37.095029 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:37.095072 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:37.095074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.095669 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.095670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:37.096244 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:37.096245 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.096523 (+ 278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.096524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:37.096558 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:37.096972 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:37.096976 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:37.098549 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:37.099793 (+ 1244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:37.099801 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:37.099804 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:37.100002 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:37.100149 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:37.100149 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:37.100149 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:37.100200 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:37.100383 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:37.100391 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:37.100442 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 15:03:37.100495 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 15:03:37.100582 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
I20260806 15:03:37.100756 12755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:37.094478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38974 (local address 127.0.236.126:39739)
0806 15:03:37.094607 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:37.094608 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:37.094629 (+ 21us) server_negotiation.cc:407] Connection header received
0806 15:03:37.094668 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:37.094669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:37.094678 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:37.094725 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:37.095125 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.095619 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.096294 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.096480 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.096520 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:37.098622 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:37.098624 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:37.098624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:37.098630 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:37.099721 (+ 1091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:37.100055 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:37.100056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:37.100056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:37.100105 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:37.100238 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:37.100238 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:37.100239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:37.100341 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:37.100382 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:37.100516 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 15:03:37.100628 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":107,"thread_start_us":83,"threads_started":1}
I20260806 15:03:37.102500 11918 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:38974:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 15:03:37.102670 11918 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:03:37.103850 12134 tablet_service.cc:1467] Processing CreateTablet for tablet 68feb7877bb3435594e847cff2110e66 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4011f5468cd7453c9d0edbc136de5a4e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 15:03:37.103871 12553 tablet_service.cc:1467] Processing CreateTablet for tablet 68feb7877bb3435594e847cff2110e66 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4011f5468cd7453c9d0edbc136de5a4e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 15:03:37.104007 12134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68feb7877bb3435594e847cff2110e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:37.104007 12553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68feb7877bb3435594e847cff2110e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:37.105427 12685 tablet_bootstrap.cc:492] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532: Bootstrap starting.
I20260806 15:03:37.105530 12344 tablet_service.cc:1467] Processing CreateTablet for tablet 68feb7877bb3435594e847cff2110e66 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4011f5468cd7453c9d0edbc136de5a4e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 15:03:37.105638 12344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68feb7877bb3435594e847cff2110e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:37.105788 12685 tablet_bootstrap.cc:654] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:37.106050 12691 tablet_bootstrap.cc:492] T 68feb7877bb3435594e847cff2110e66 P b584a933f35a49678c7f0adde4e4c211: Bootstrap starting.
I20260806 15:03:37.106377 12691 tablet_bootstrap.cc:654] T 68feb7877bb3435594e847cff2110e66 P b584a933f35a49678c7f0adde4e4c211: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:37.106524 12685 tablet_bootstrap.cc:492] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532: No bootstrap required, opened a new log
I20260806 15:03:37.106580 12685 ts_tablet_manager.cc:1397] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 15:03:37.106706 12685 raft_consensus.cc:348] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.106782 12685 raft_consensus.cc:374] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:37.106801 12685 raft_consensus.cc:729] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 560d593b247846858b20f9d7ea174532, State: Initialized, Role: FOLLOWER
I20260806 15:03:37.106843 12685 consensus_queue.cc:260] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.106945 12685 ts_tablet_manager.cc:1428] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 15:03:37.107004 12691 tablet_bootstrap.cc:492] T 68feb7877bb3435594e847cff2110e66 P b584a933f35a49678c7f0adde4e4c211: No bootstrap required, opened a new log
I20260806 15:03:37.107046 12691 ts_tablet_manager.cc:1397] T 68feb7877bb3435594e847cff2110e66 P b584a933f35a49678c7f0adde4e4c211: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:37.107054 12693 tablet_replica.cc:439] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } } }
I20260806 15:03:37.107117 12693 tablet_replica.cc:442] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:03:37.107194 12687 tablet_bootstrap.cc:492] T 68feb7877bb3435594e847cff2110e66 P c4eb9691b8c3440c925e003fe1a311f5: Bootstrap starting.
I20260806 15:03:37.107282 12691 raft_consensus.cc:348] T 68feb7877bb3435594e847cff2110e66 P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.107347 12691 raft_consensus.cc:374] T 68feb7877bb3435594e847cff2110e66 P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:37.107365 12691 raft_consensus.cc:729] T 68feb7877bb3435594e847cff2110e66 P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b584a933f35a49678c7f0adde4e4c211, State: Initialized, Role: FOLLOWER
I20260806 15:03:37.107420 12691 consensus_queue.cc:260] T 68feb7877bb3435594e847cff2110e66 P b584a933f35a49678c7f0adde4e4c211 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.107511 12691 ts_tablet_manager.cc:1428] T 68feb7877bb3435594e847cff2110e66 P b584a933f35a49678c7f0adde4e4c211: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:37.107568 12695 tablet_replica.cc:439] T 68feb7877bb3435594e847cff2110e66 P b584a933f35a49678c7f0adde4e4c211: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } } }
I20260806 15:03:37.107628 12687 tablet_bootstrap.cc:654] T 68feb7877bb3435594e847cff2110e66 P c4eb9691b8c3440c925e003fe1a311f5: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:37.107652 12695 tablet_replica.cc:442] T 68feb7877bb3435594e847cff2110e66 P b584a933f35a49678c7f0adde4e4c211: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:03:37.108227 12687 tablet_bootstrap.cc:492] T 68feb7877bb3435594e847cff2110e66 P c4eb9691b8c3440c925e003fe1a311f5: No bootstrap required, opened a new log
I20260806 15:03:37.108279 12687 ts_tablet_manager.cc:1397] T 68feb7877bb3435594e847cff2110e66 P c4eb9691b8c3440c925e003fe1a311f5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:37.108422 12687 raft_consensus.cc:348] T 68feb7877bb3435594e847cff2110e66 P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.108479 12687 raft_consensus.cc:374] T 68feb7877bb3435594e847cff2110e66 P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:37.108490 12687 raft_consensus.cc:729] T 68feb7877bb3435594e847cff2110e66 P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4eb9691b8c3440c925e003fe1a311f5, State: Initialized, Role: FOLLOWER
I20260806 15:03:37.108520 12687 consensus_queue.cc:260] T 68feb7877bb3435594e847cff2110e66 P c4eb9691b8c3440c925e003fe1a311f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.108582 12687 ts_tablet_manager.cc:1428] T 68feb7877bb3435594e847cff2110e66 P c4eb9691b8c3440c925e003fe1a311f5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:37.108640 12692 tablet_replica.cc:439] T 68feb7877bb3435594e847cff2110e66 P c4eb9691b8c3440c925e003fe1a311f5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } } }
I20260806 15:03:37.108747 12692 tablet_replica.cc:442] T 68feb7877bb3435594e847cff2110e66 P c4eb9691b8c3440c925e003fe1a311f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:03:37.231902 12693 raft_consensus.cc:482] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:03:37.231990 12693 raft_consensus.cc:504] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.232121 12693 leader_election.cc:288] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b584a933f35a49678c7f0adde4e4c211 (127.0.236.65:46041), c4eb9691b8c3440c925e003fe1a311f5 (127.0.236.66:44243)
I20260806 15:03:37.232292 12165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68feb7877bb3435594e847cff2110e66" candidate_uuid: "560d593b247846858b20f9d7ea174532" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b584a933f35a49678c7f0adde4e4c211" is_pre_election: true
I20260806 15:03:37.232290 12376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68feb7877bb3435594e847cff2110e66" candidate_uuid: "560d593b247846858b20f9d7ea174532" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4eb9691b8c3440c925e003fe1a311f5" is_pre_election: true
I20260806 15:03:37.232378 12165 raft_consensus.cc:2454] T 68feb7877bb3435594e847cff2110e66 P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 560d593b247846858b20f9d7ea174532 in term 0.
I20260806 15:03:37.232381 12376 raft_consensus.cc:2454] T 68feb7877bb3435594e847cff2110e66 P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 560d593b247846858b20f9d7ea174532 in term 0.
I20260806 15:03:37.232508 12482 leader_election.cc:302] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 560d593b247846858b20f9d7ea174532, b584a933f35a49678c7f0adde4e4c211; no voters:
I20260806 15:03:37.232604 12693 raft_consensus.cc:2781] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:03:37.232643 12693 raft_consensus.cc:482] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:03:37.232662 12693 raft_consensus.cc:3037] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:37.233250 12693 raft_consensus.cc:504] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.233371 12693 leader_election.cc:288] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532 [CANDIDATE]: Term 1 election: Requested vote from peers b584a933f35a49678c7f0adde4e4c211 (127.0.236.65:46041), c4eb9691b8c3440c925e003fe1a311f5 (127.0.236.66:44243)
I20260806 15:03:37.233520 12165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68feb7877bb3435594e847cff2110e66" candidate_uuid: "560d593b247846858b20f9d7ea174532" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b584a933f35a49678c7f0adde4e4c211"
I20260806 15:03:37.233525 12376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68feb7877bb3435594e847cff2110e66" candidate_uuid: "560d593b247846858b20f9d7ea174532" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4eb9691b8c3440c925e003fe1a311f5"
I20260806 15:03:37.233584 12165 raft_consensus.cc:3037] T 68feb7877bb3435594e847cff2110e66 P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:37.233587 12376 raft_consensus.cc:3037] T 68feb7877bb3435594e847cff2110e66 P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:37.234166 12165 raft_consensus.cc:2454] T 68feb7877bb3435594e847cff2110e66 P b584a933f35a49678c7f0adde4e4c211 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 560d593b247846858b20f9d7ea174532 in term 1.
I20260806 15:03:37.234187 12376 raft_consensus.cc:2454] T 68feb7877bb3435594e847cff2110e66 P c4eb9691b8c3440c925e003fe1a311f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 560d593b247846858b20f9d7ea174532 in term 1.
I20260806 15:03:37.234335 12484 leader_election.cc:302] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 560d593b247846858b20f9d7ea174532, c4eb9691b8c3440c925e003fe1a311f5; no voters:
I20260806 15:03:37.234457 12693 raft_consensus.cc:2781] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:37.234501 12693 raft_consensus.cc:686] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532 [term 1 LEADER]: Becoming Leader. State: Replica: 560d593b247846858b20f9d7ea174532, State: Running, Role: LEADER
I20260806 15:03:37.234542 12693 consensus_queue.cc:237] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.234797 12700 tablet_replica.cc:439] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532: TxnStatusTablet state changed. Reason: New leader 560d593b247846858b20f9d7ea174532. Latest consensus state: current_term: 1 leader_uuid: "560d593b247846858b20f9d7ea174532" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } } }
I20260806 15:03:37.234886 12700 tablet_replica.cc:442] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532: This TxnStatusTablet replica's current role is: LEADER
I20260806 15:03:37.235036 12760 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 15:03:37.235162 12760 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 15:03:37.235096 11918 catalog_manager.cc:5495] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532 reported cstate change: term changed from 0 to 1, leader changed from <none> to 560d593b247846858b20f9d7ea174532 (127.0.236.67). New cstate: current_term: 1 leader_uuid: "560d593b247846858b20f9d7ea174532" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } health_report { overall_health: HEALTHY } } }
I20260806 15:03:37.250531 12684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:37.248365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55342 (local address 127.0.236.67:45805)
0806 15:03:37.248404 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:37.248407 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:37.248416 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:03:37.248484 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:37.248485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:37.248493 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:37.248545 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:03:37.248678 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.249150 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.249866 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.250101 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.250157 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:37.250308 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:03:37.250344 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:03:37.250384 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:37.250441 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 15:03:37.250466 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:03:37.250560 12753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:37.248306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:45805 (local address 127.0.0.1:55342)
0806 15:03:37.248368 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:37.248378 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:37.248416 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:37.248569 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:37.248571 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:37.248572 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 15:03:37.248622 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:37.248625 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.249219 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.249220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:37.249792 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:37.249794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.250164 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.250165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:37.250224 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:37.250241 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 15:03:37.250391 (+ 150us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 15:03:37.250399 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 15:03:37.250446 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 15:03:37.250492 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 15:03:37.251426 12165 raft_consensus.cc:1261] T 68feb7877bb3435594e847cff2110e66 P b584a933f35a49678c7f0adde4e4c211 [term 1 FOLLOWER]: Refusing update from remote peer 560d593b247846858b20f9d7ea174532: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:37.251435 12376 raft_consensus.cc:1261] T 68feb7877bb3435594e847cff2110e66 P c4eb9691b8c3440c925e003fe1a311f5 [term 1 FOLLOWER]: Refusing update from remote peer 560d593b247846858b20f9d7ea174532: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:37.251642 12700 consensus_queue.cc:1035] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:37.251744 12693 consensus_queue.cc:1035] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:37.252193 12695 tablet_replica.cc:439] T 68feb7877bb3435594e847cff2110e66 P b584a933f35a49678c7f0adde4e4c211: TxnStatusTablet state changed. Reason: New leader 560d593b247846858b20f9d7ea174532. Latest consensus state: current_term: 1 leader_uuid: "560d593b247846858b20f9d7ea174532" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } } }
I20260806 15:03:37.252206 12692 tablet_replica.cc:439] T 68feb7877bb3435594e847cff2110e66 P c4eb9691b8c3440c925e003fe1a311f5: TxnStatusTablet state changed. Reason: New leader 560d593b247846858b20f9d7ea174532. Latest consensus state: current_term: 1 leader_uuid: "560d593b247846858b20f9d7ea174532" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } } }
I20260806 15:03:37.252275 12692 tablet_replica.cc:442] T 68feb7877bb3435594e847cff2110e66 P c4eb9691b8c3440c925e003fe1a311f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:03:37.252275 12695 tablet_replica.cc:442] T 68feb7877bb3435594e847cff2110e66 P b584a933f35a49678c7f0adde4e4c211: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:03:37.253131 12700 tablet_replica.cc:439] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "560d593b247846858b20f9d7ea174532" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } } }
I20260806 15:03:37.253206 12700 tablet_replica.cc:442] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532: This TxnStatusTablet replica's current role is: LEADER
I20260806 15:03:37.253206 12695 tablet_replica.cc:439] T 68feb7877bb3435594e847cff2110e66 P b584a933f35a49678c7f0adde4e4c211: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "560d593b247846858b20f9d7ea174532" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } } }
I20260806 15:03:37.253229 12771 tablet_replica.cc:439] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "560d593b247846858b20f9d7ea174532" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } } }
I20260806 15:03:37.253278 12695 tablet_replica.cc:442] T 68feb7877bb3435594e847cff2110e66 P b584a933f35a49678c7f0adde4e4c211: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:03:37.253293 12771 tablet_replica.cc:442] T 68feb7877bb3435594e847cff2110e66 P 560d593b247846858b20f9d7ea174532: This TxnStatusTablet replica's current role is: LEADER
I20260806 15:03:37.254148 12692 tablet_replica.cc:439] T 68feb7877bb3435594e847cff2110e66 P c4eb9691b8c3440c925e003fe1a311f5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "560d593b247846858b20f9d7ea174532" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } } }
I20260806 15:03:37.254231 12692 tablet_replica.cc:442] T 68feb7877bb3435594e847cff2110e66 P c4eb9691b8c3440c925e003fe1a311f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:03:37.258045 12776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:37.255505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39006 (local address 127.0.236.65:46041)
0806 15:03:37.255600 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:37.255602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:37.255609 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:03:37.255644 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:37.255645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:37.255662 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:37.255725 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:03:37.255845 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.256300 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.257314 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.257531 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.257581 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:37.257724 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:03:37.257815 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:03:37.257867 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:37.257964 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 15:03:37.257982 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":55,"threads_started":1}
I20260806 15:03:37.262657 12784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:37.260168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:45805 (local address 127.0.236.65:60035)
0806 15:03:37.260335 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:37.260344 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:37.260391 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:37.260500 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:37.260501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:37.260502 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 15:03:37.260550 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:37.260551 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.261341 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.261341 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:37.261918 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:37.261920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.262344 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.262345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:37.262391 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:37.262404 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 15:03:37.262538 (+ 134us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 15:03:37.262539 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 15:03:37.262567 (+ 28us) client_negotiation.cc:240] Negotiation successful
0806 15:03:37.262594 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260806 15:03:37.262657 12684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:37.260246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:60035 (local address 127.0.236.67:45805)
0806 15:03:37.260283 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:37.260285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:37.260393 (+ 108us) server_negotiation.cc:407] Connection header received
0806 15:03:37.260428 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:37.260429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:37.260437 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:37.260498 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:03:37.260585 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.261280 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.261986 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.262281 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.262340 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:37.262461 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:03:37.262491 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:03:37.262538 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:37.262571 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 15:03:37.262591 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 15:03:37.264488 12580 raft_consensus.cc:1261] T e572cb484ed34b08b116bd534571e61c P 560d593b247846858b20f9d7ea174532 [term 1 FOLLOWER]: Refusing update from remote peer b584a933f35a49678c7f0adde4e4c211: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:37.264531 12376 raft_consensus.cc:1261] T e572cb484ed34b08b116bd534571e61c P c4eb9691b8c3440c925e003fe1a311f5 [term 1 FOLLOWER]: Refusing update from remote peer b584a933f35a49678c7f0adde4e4c211: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:37.264737 12742 consensus_queue.cc:1035] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:37.264850 12695 consensus_queue.cc:1035] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211 [LEADER]: Connected to new peer: Peer: permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:37.272202 12776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:37.269823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:38705 (local address 127.0.236.65:46041)
0806 15:03:37.269859 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:37.269861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:37.270016 (+ 155us) server_negotiation.cc:407] Connection header received
0806 15:03:37.270108 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:37.270109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:37.270118 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:37.270145 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:03:37.270362 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.270790 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.271484 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.271728 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.271792 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:37.272026 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:03:37.272053 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:03:37.272090 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:37.272142 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 15:03:37.272157 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 15:03:37.272286 12792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:37.269784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:46041 (local address 127.0.236.67:38705)
0806 15:03:37.269956 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:37.269966 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:37.270040 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:37.270197 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:37.270198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:37.270199 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 15:03:37.270297 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:37.270300 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.270854 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.270855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:37.271412 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:37.271413 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.271801 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.271802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:37.271901 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:37.271923 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 15:03:37.272094 (+ 171us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 15:03:37.272095 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 15:03:37.272139 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 15:03:37.272179 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":76,"threads_started":1}
I20260806 15:03:37.281338 11918 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:37.281425 11918 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:37.282130 11918 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef: Sending DeleteTablet for 3 replicas of tablet e572cb484ed34b08b116bd534571e61c
I20260806 15:03:37.282431 12134 tablet_service.cc:1514] Processing DeleteTablet for tablet e572cb484ed34b08b116bd534571e61c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:37 UTC) from {username='oryx', principal='oryx/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:39000
I20260806 15:03:37.282521 12344 tablet_service.cc:1514] Processing DeleteTablet for tablet e572cb484ed34b08b116bd534571e61c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:37 UTC) from {username='oryx', principal='oryx/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:52844
I20260806 15:03:37.282598 12734 tablet_replica.cc:330] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211: stopping tablet replica
I20260806 15:03:37.282647 12734 raft_consensus.cc:2229] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211 [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:03:37.282721 12734 raft_consensus.cc:2258] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:37.282918 12728 tablet_replica.cc:330] T e572cb484ed34b08b116bd534571e61c P c4eb9691b8c3440c925e003fe1a311f5: stopping tablet replica
I20260806 15:03:37.282980 12728 raft_consensus.cc:2229] T e572cb484ed34b08b116bd534571e61c P c4eb9691b8c3440c925e003fe1a311f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:37.283020 12728 raft_consensus.cc:2258] T e572cb484ed34b08b116bd534571e61c P c4eb9691b8c3440c925e003fe1a311f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:37.283373 12728 ts_tablet_manager.cc:1905] T e572cb484ed34b08b116bd534571e61c P c4eb9691b8c3440c925e003fe1a311f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:37.284965 12728 ts_tablet_manager.cc:1918] T e572cb484ed34b08b116bd534571e61c P c4eb9691b8c3440c925e003fe1a311f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:03:37.285015 12728 log.cc:1196] T e572cb484ed34b08b116bd534571e61c P c4eb9691b8c3440c925e003fe1a311f5: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-1/wal/wals/e572cb484ed34b08b116bd534571e61c
I20260806 15:03:37.285223 12728 ts_tablet_manager.cc:1939] T e572cb484ed34b08b116bd534571e61c P c4eb9691b8c3440c925e003fe1a311f5: Deleting consensus metadata
I20260806 15:03:37.285507 11899 catalog_manager.cc:4841] TS c4eb9691b8c3440c925e003fe1a311f5 (127.0.236.66:44243): tablet e572cb484ed34b08b116bd534571e61c (table test-table [id=406e61bb3a78400699d5be8f2d6bea3c]) successfully deleted
I20260806 15:03:37.285652 12553 tablet_service.cc:1514] Processing DeleteTablet for tablet e572cb484ed34b08b116bd534571e61c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:37 UTC) from {username='oryx', principal='oryx/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:55312
I20260806 15:03:37.285740 12729 tablet_replica.cc:330] T e572cb484ed34b08b116bd534571e61c P 560d593b247846858b20f9d7ea174532: stopping tablet replica
I20260806 15:03:37.285784 12729 raft_consensus.cc:2229] T e572cb484ed34b08b116bd534571e61c P 560d593b247846858b20f9d7ea174532 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:37.285812 12729 raft_consensus.cc:2258] T e572cb484ed34b08b116bd534571e61c P 560d593b247846858b20f9d7ea174532 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:37.285838 12734 ts_tablet_manager.cc:1905] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:37.287323 12734 ts_tablet_manager.cc:1918] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:03:37.287384 12734 log.cc:1196] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-0/wal/wals/e572cb484ed34b08b116bd534571e61c
I20260806 15:03:37.287570 12734 ts_tablet_manager.cc:1939] T e572cb484ed34b08b116bd534571e61c P b584a933f35a49678c7f0adde4e4c211: Deleting consensus metadata
I20260806 15:03:37.287868 11895 catalog_manager.cc:4841] TS b584a933f35a49678c7f0adde4e4c211 (127.0.236.65:46041): tablet e572cb484ed34b08b116bd534571e61c (table test-table [id=406e61bb3a78400699d5be8f2d6bea3c]) successfully deleted
I20260806 15:03:37.287974 12729 ts_tablet_manager.cc:1905] T e572cb484ed34b08b116bd534571e61c P 560d593b247846858b20f9d7ea174532: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:37.289362 12729 ts_tablet_manager.cc:1918] T e572cb484ed34b08b116bd534571e61c P 560d593b247846858b20f9d7ea174532: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:03:37.289423 12729 log.cc:1196] T e572cb484ed34b08b116bd534571e61c P 560d593b247846858b20f9d7ea174532: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/e572cb484ed34b08b116bd534571e61c
I20260806 15:03:37.289611 12729 ts_tablet_manager.cc:1939] T e572cb484ed34b08b116bd534571e61c P 560d593b247846858b20f9d7ea174532: Deleting consensus metadata
I20260806 15:03:37.289929 11895 catalog_manager.cc:4841] TS 560d593b247846858b20f9d7ea174532 (127.0.236.67:45805): tablet e572cb484ed34b08b116bd534571e61c (table test-table [id=406e61bb3a78400699d5be8f2d6bea3c]) successfully deleted
Aug 06 15:03:37 dist-test-slave-tc2s krb5kdc[11848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 15:03:37 dist-test-slave-tc2s krb5kdc[11848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:37 dist-test-slave-tc2s krb5kdc[11848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 15:03:37.319713 12755 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:37.312968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38990 (local address 127.0.236.126:39739)
0806 15:03:37.313030 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:37.313032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:37.313215 (+ 183us) server_negotiation.cc:407] Connection header received
0806 15:03:37.313273 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:37.313274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:37.313290 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:37.313356 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:37.314073 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.314824 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.315926 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.316222 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.316297 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:37.319644 (+ 3347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38990: BlockingRecv error: recv got EOF from 127.0.0.1:38990 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 06 15:03:37 dist-test-slave-tc2s krb5kdc[11848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM
I20260806 15:03:37.328912 12755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:37.320686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39006 (local address 127.0.236.126:39739)
0806 15:03:37.320743 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:37.320745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:37.320836 (+ 91us) server_negotiation.cc:407] Connection header received
0806 15:03:37.320884 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:37.320885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:37.320894 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:37.320959 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:37.321492 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.322281 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.323242 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.323538 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.323604 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:37.326106 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:37.326107 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:37.326108 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:37.326115 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:37.327743 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:37.328144 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:37.328145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:37.328146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:37.328210 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:37.328397 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:37.328398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:37.328398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:37.328538 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:37.328646 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:37.328743 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 15:03:37.328821 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 15:03:37.330847 11918 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39006:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:03:37.331032 11918 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:03:37.332356 12344 tablet_service.cc:1467] Processing CreateTablet for tablet 03869e82761240119fafcfa9afa6122d (DEFAULT_TABLE table=test-table [id=43440bb86aa74bf69e11d15abcd399fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:37.332399 12134 tablet_service.cc:1467] Processing CreateTablet for tablet 03869e82761240119fafcfa9afa6122d (DEFAULT_TABLE table=test-table [id=43440bb86aa74bf69e11d15abcd399fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:37.332507 12344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03869e82761240119fafcfa9afa6122d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:37.332525 12134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03869e82761240119fafcfa9afa6122d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:37.333822 12553 tablet_service.cc:1467] Processing CreateTablet for tablet 03869e82761240119fafcfa9afa6122d (DEFAULT_TABLE table=test-table [id=43440bb86aa74bf69e11d15abcd399fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:37.333952 12687 tablet_bootstrap.cc:492] T 03869e82761240119fafcfa9afa6122d P c4eb9691b8c3440c925e003fe1a311f5: Bootstrap starting.
I20260806 15:03:37.333967 12553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03869e82761240119fafcfa9afa6122d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:37.334259 12687 tablet_bootstrap.cc:654] T 03869e82761240119fafcfa9afa6122d P c4eb9691b8c3440c925e003fe1a311f5: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:37.334797 12691 tablet_bootstrap.cc:492] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211: Bootstrap starting.
I20260806 15:03:37.335160 12685 tablet_bootstrap.cc:492] T 03869e82761240119fafcfa9afa6122d P 560d593b247846858b20f9d7ea174532: Bootstrap starting.
I20260806 15:03:37.335394 12691 tablet_bootstrap.cc:654] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:37.335445 12685 tablet_bootstrap.cc:654] T 03869e82761240119fafcfa9afa6122d P 560d593b247846858b20f9d7ea174532: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:37.336328 12687 tablet_bootstrap.cc:492] T 03869e82761240119fafcfa9afa6122d P c4eb9691b8c3440c925e003fe1a311f5: No bootstrap required, opened a new log
I20260806 15:03:37.336334 12691 tablet_bootstrap.cc:492] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211: No bootstrap required, opened a new log
I20260806 15:03:37.336377 12691 ts_tablet_manager.cc:1397] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:03:37.336375 12687 ts_tablet_manager.cc:1397] T 03869e82761240119fafcfa9afa6122d P c4eb9691b8c3440c925e003fe1a311f5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:03:37.336515 12691 raft_consensus.cc:348] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.336514 12687 raft_consensus.cc:348] T 03869e82761240119fafcfa9afa6122d P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.336580 12687 raft_consensus.cc:374] T 03869e82761240119fafcfa9afa6122d P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:37.336580 12691 raft_consensus.cc:374] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:37.336602 12691 raft_consensus.cc:729] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b584a933f35a49678c7f0adde4e4c211, State: Initialized, Role: FOLLOWER
I20260806 15:03:37.336601 12687 raft_consensus.cc:729] T 03869e82761240119fafcfa9afa6122d P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4eb9691b8c3440c925e003fe1a311f5, State: Initialized, Role: FOLLOWER
I20260806 15:03:37.336642 12687 consensus_queue.cc:260] T 03869e82761240119fafcfa9afa6122d P c4eb9691b8c3440c925e003fe1a311f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.336648 12691 consensus_queue.cc:260] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.336722 12687 ts_tablet_manager.cc:1428] T 03869e82761240119fafcfa9afa6122d P c4eb9691b8c3440c925e003fe1a311f5: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:37.336732 12691 ts_tablet_manager.cc:1428] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:37.338261 12685 tablet_bootstrap.cc:492] T 03869e82761240119fafcfa9afa6122d P 560d593b247846858b20f9d7ea174532: No bootstrap required, opened a new log
I20260806 15:03:37.338320 12685 ts_tablet_manager.cc:1397] T 03869e82761240119fafcfa9afa6122d P 560d593b247846858b20f9d7ea174532: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20260806 15:03:37.338406 12685 raft_consensus.cc:348] T 03869e82761240119fafcfa9afa6122d P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.338470 12685 raft_consensus.cc:374] T 03869e82761240119fafcfa9afa6122d P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:37.338490 12685 raft_consensus.cc:729] T 03869e82761240119fafcfa9afa6122d P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 560d593b247846858b20f9d7ea174532, State: Initialized, Role: FOLLOWER
I20260806 15:03:37.338532 12685 consensus_queue.cc:260] T 03869e82761240119fafcfa9afa6122d P 560d593b247846858b20f9d7ea174532 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.338613 12685 ts_tablet_manager.cc:1428] T 03869e82761240119fafcfa9afa6122d P 560d593b247846858b20f9d7ea174532: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 15:03:37.499660 12788 raft_consensus.cc:482] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:03:37.499768 12788 raft_consensus.cc:504] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.499956 12788 leader_election.cc:288] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4eb9691b8c3440c925e003fe1a311f5 (127.0.236.66:44243), 560d593b247846858b20f9d7ea174532 (127.0.236.67:45805)
I20260806 15:03:37.500181 12580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03869e82761240119fafcfa9afa6122d" candidate_uuid: "b584a933f35a49678c7f0adde4e4c211" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "560d593b247846858b20f9d7ea174532" is_pre_election: true
I20260806 15:03:37.500177 12376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03869e82761240119fafcfa9afa6122d" candidate_uuid: "b584a933f35a49678c7f0adde4e4c211" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4eb9691b8c3440c925e003fe1a311f5" is_pre_election: true
I20260806 15:03:37.500278 12580 raft_consensus.cc:2454] T 03869e82761240119fafcfa9afa6122d P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b584a933f35a49678c7f0adde4e4c211 in term 0.
I20260806 15:03:37.500281 12376 raft_consensus.cc:2454] T 03869e82761240119fafcfa9afa6122d P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b584a933f35a49678c7f0adde4e4c211 in term 0.
I20260806 15:03:37.500470 12057 leader_election.cc:302] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 560d593b247846858b20f9d7ea174532, b584a933f35a49678c7f0adde4e4c211; no voters:
I20260806 15:03:37.500582 12788 raft_consensus.cc:2781] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:03:37.500636 12788 raft_consensus.cc:482] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:03:37.500650 12788 raft_consensus.cc:3037] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:37.501410 12788 raft_consensus.cc:504] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.501539 12788 leader_election.cc:288] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211 [CANDIDATE]: Term 1 election: Requested vote from peers c4eb9691b8c3440c925e003fe1a311f5 (127.0.236.66:44243), 560d593b247846858b20f9d7ea174532 (127.0.236.67:45805)
I20260806 15:03:37.501660 12376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03869e82761240119fafcfa9afa6122d" candidate_uuid: "b584a933f35a49678c7f0adde4e4c211" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4eb9691b8c3440c925e003fe1a311f5"
I20260806 15:03:37.501741 12376 raft_consensus.cc:3037] T 03869e82761240119fafcfa9afa6122d P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:37.501709 12580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03869e82761240119fafcfa9afa6122d" candidate_uuid: "b584a933f35a49678c7f0adde4e4c211" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "560d593b247846858b20f9d7ea174532"
I20260806 15:03:37.501771 12580 raft_consensus.cc:3037] T 03869e82761240119fafcfa9afa6122d P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:37.502532 12376 raft_consensus.cc:2454] T 03869e82761240119fafcfa9afa6122d P c4eb9691b8c3440c925e003fe1a311f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b584a933f35a49678c7f0adde4e4c211 in term 1.
I20260806 15:03:37.502599 12580 raft_consensus.cc:2454] T 03869e82761240119fafcfa9afa6122d P 560d593b247846858b20f9d7ea174532 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b584a933f35a49678c7f0adde4e4c211 in term 1.
I20260806 15:03:37.502731 12064 leader_election.cc:302] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b584a933f35a49678c7f0adde4e4c211, c4eb9691b8c3440c925e003fe1a311f5; no voters:
I20260806 15:03:37.502847 12788 raft_consensus.cc:2781] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:37.502899 12788 raft_consensus.cc:686] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211 [term 1 LEADER]: Becoming Leader. State: Replica: b584a933f35a49678c7f0adde4e4c211, State: Running, Role: LEADER
I20260806 15:03:37.502995 12788 consensus_queue.cc:237] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.503542 11918 catalog_manager.cc:5495] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211 reported cstate change: term changed from 0 to 1, leader changed from <none> to b584a933f35a49678c7f0adde4e4c211 (127.0.236.65). New cstate: current_term: 1 leader_uuid: "b584a933f35a49678c7f0adde4e4c211" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } health_report { overall_health: UNKNOWN } } }
I20260806 15:03:37.519215 12776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:37.517060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39012 (local address 127.0.236.65:46041)
0806 15:03:37.517102 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:37.517105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:37.517114 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:03:37.517168 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:37.517169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:37.517181 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:37.517226 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:03:37.517344 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.517821 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.518606 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.518794 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.518848 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:37.518989 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:03:37.519020 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:03:37.519060 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:37.519136 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 15:03:37.519163 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 15:03:37.519879 12376 raft_consensus.cc:1261] T 03869e82761240119fafcfa9afa6122d P c4eb9691b8c3440c925e003fe1a311f5 [term 1 FOLLOWER]: Refusing update from remote peer b584a933f35a49678c7f0adde4e4c211: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:37.519909 12580 raft_consensus.cc:1261] T 03869e82761240119fafcfa9afa6122d P 560d593b247846858b20f9d7ea174532 [term 1 FOLLOWER]: Refusing update from remote peer b584a933f35a49678c7f0adde4e4c211: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:37.520112 12695 consensus_queue.cc:1035] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:37.520211 12788 consensus_queue.cc:1035] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211 [LEADER]: Connected to new peer: Peer: permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:37.522500 11918 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:37.522578 11918 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:37.523087 11918 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef: Sending DeleteTablet for 3 replicas of tablet 03869e82761240119fafcfa9afa6122d
I20260806 15:03:37.523289 12134 tablet_service.cc:1514] Processing DeleteTablet for tablet 03869e82761240119fafcfa9afa6122d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:37 UTC) from {username='oryx', principal='oryx/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:39000
I20260806 15:03:37.523320 12344 tablet_service.cc:1514] Processing DeleteTablet for tablet 03869e82761240119fafcfa9afa6122d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:37 UTC) from {username='oryx', principal='oryx/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:52844
I20260806 15:03:37.523371 12553 tablet_service.cc:1514] Processing DeleteTablet for tablet 03869e82761240119fafcfa9afa6122d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:37 UTC) from {username='oryx', principal='oryx/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:55312
I20260806 15:03:37.523417 12734 tablet_replica.cc:330] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211: stopping tablet replica
I20260806 15:03:37.523417 12728 tablet_replica.cc:330] T 03869e82761240119fafcfa9afa6122d P c4eb9691b8c3440c925e003fe1a311f5: stopping tablet replica
I20260806 15:03:37.523465 12728 raft_consensus.cc:2229] T 03869e82761240119fafcfa9afa6122d P c4eb9691b8c3440c925e003fe1a311f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:37.523466 12734 raft_consensus.cc:2229] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211 [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:03:37.523510 12728 raft_consensus.cc:2258] T 03869e82761240119fafcfa9afa6122d P c4eb9691b8c3440c925e003fe1a311f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:37.523518 12734 raft_consensus.cc:2258] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:37.523586 12729 tablet_replica.cc:330] T 03869e82761240119fafcfa9afa6122d P 560d593b247846858b20f9d7ea174532: stopping tablet replica
I20260806 15:03:37.523557 11918 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39006:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 15:03:37.523654 12729 raft_consensus.cc:2229] T 03869e82761240119fafcfa9afa6122d P 560d593b247846858b20f9d7ea174532 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:37.523687 12729 raft_consensus.cc:2258] T 03869e82761240119fafcfa9afa6122d P 560d593b247846858b20f9d7ea174532 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 15:03:37.523689 11918 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:03:37.523939 12734 ts_tablet_manager.cc:1905] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:37.523959 12728 ts_tablet_manager.cc:1905] T 03869e82761240119fafcfa9afa6122d P c4eb9691b8c3440c925e003fe1a311f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:37.524752 12553 tablet_service.cc:1467] Processing CreateTablet for tablet 671beac9ff204bce85b312bd94870ebf (DEFAULT_TABLE table=test-table [id=485431eba42242febdee23382cd14d86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:37.524752 12344 tablet_service.cc:1467] Processing CreateTablet for tablet 671beac9ff204bce85b312bd94870ebf (DEFAULT_TABLE table=test-table [id=485431eba42242febdee23382cd14d86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:37.524863 12553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 671beac9ff204bce85b312bd94870ebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:37.524863 12344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 671beac9ff204bce85b312bd94870ebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:37.525431 12728 ts_tablet_manager.cc:1918] T 03869e82761240119fafcfa9afa6122d P c4eb9691b8c3440c925e003fe1a311f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:03:37.525482 12728 log.cc:1196] T 03869e82761240119fafcfa9afa6122d P c4eb9691b8c3440c925e003fe1a311f5: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-1/wal/wals/03869e82761240119fafcfa9afa6122d
I20260806 15:03:37.525528 12729 ts_tablet_manager.cc:1905] T 03869e82761240119fafcfa9afa6122d P 560d593b247846858b20f9d7ea174532: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:37.525681 12728 ts_tablet_manager.cc:1939] T 03869e82761240119fafcfa9afa6122d P c4eb9691b8c3440c925e003fe1a311f5: Deleting consensus metadata
I20260806 15:03:37.526170 12687 tablet_bootstrap.cc:492] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5: Bootstrap starting.
I20260806 15:03:37.526194 12685 tablet_bootstrap.cc:492] T 671beac9ff204bce85b312bd94870ebf P 560d593b247846858b20f9d7ea174532: Bootstrap starting.
I20260806 15:03:37.526278 11899 catalog_manager.cc:4841] TS c4eb9691b8c3440c925e003fe1a311f5 (127.0.236.66:44243): tablet 03869e82761240119fafcfa9afa6122d (table test-table [id=43440bb86aa74bf69e11d15abcd399fc]) successfully deleted
I20260806 15:03:37.526510 12685 tablet_bootstrap.cc:654] T 671beac9ff204bce85b312bd94870ebf P 560d593b247846858b20f9d7ea174532: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:37.526520 12734 ts_tablet_manager.cc:1918] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:03:37.526531 12687 tablet_bootstrap.cc:654] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:37.526566 12734 log.cc:1196] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-0/wal/wals/03869e82761240119fafcfa9afa6122d
I20260806 15:03:37.526777 12734 ts_tablet_manager.cc:1939] T 03869e82761240119fafcfa9afa6122d P b584a933f35a49678c7f0adde4e4c211: Deleting consensus metadata
I20260806 15:03:37.526906 12729 ts_tablet_manager.cc:1918] T 03869e82761240119fafcfa9afa6122d P 560d593b247846858b20f9d7ea174532: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:03:37.526948 12729 log.cc:1196] T 03869e82761240119fafcfa9afa6122d P 560d593b247846858b20f9d7ea174532: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/03869e82761240119fafcfa9afa6122d
I20260806 15:03:37.527037 12134 tablet_service.cc:1467] Processing CreateTablet for tablet 671beac9ff204bce85b312bd94870ebf (DEFAULT_TABLE table=test-table [id=485431eba42242febdee23382cd14d86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:37.527077 11895 catalog_manager.cc:4841] TS b584a933f35a49678c7f0adde4e4c211 (127.0.236.65:46041): tablet 03869e82761240119fafcfa9afa6122d (table test-table [id=43440bb86aa74bf69e11d15abcd399fc]) successfully deleted
I20260806 15:03:37.527125 12729 ts_tablet_manager.cc:1939] T 03869e82761240119fafcfa9afa6122d P 560d593b247846858b20f9d7ea174532: Deleting consensus metadata
I20260806 15:03:37.527165 12134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 671beac9ff204bce85b312bd94870ebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:37.527509 11895 catalog_manager.cc:4841] TS 560d593b247846858b20f9d7ea174532 (127.0.236.67:45805): tablet 03869e82761240119fafcfa9afa6122d (table test-table [id=43440bb86aa74bf69e11d15abcd399fc]) successfully deleted
I20260806 15:03:37.527639 12685 tablet_bootstrap.cc:492] T 671beac9ff204bce85b312bd94870ebf P 560d593b247846858b20f9d7ea174532: No bootstrap required, opened a new log
I20260806 15:03:37.527685 12685 ts_tablet_manager.cc:1397] T 671beac9ff204bce85b312bd94870ebf P 560d593b247846858b20f9d7ea174532: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260806 15:03:37.527783 12685 raft_consensus.cc:348] T 671beac9ff204bce85b312bd94870ebf P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.527837 12685 raft_consensus.cc:374] T 671beac9ff204bce85b312bd94870ebf P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:37.527849 12685 raft_consensus.cc:729] T 671beac9ff204bce85b312bd94870ebf P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 560d593b247846858b20f9d7ea174532, State: Initialized, Role: FOLLOWER
I20260806 15:03:37.527879 12685 consensus_queue.cc:260] T 671beac9ff204bce85b312bd94870ebf P 560d593b247846858b20f9d7ea174532 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.527969 12685 ts_tablet_manager.cc:1428] T 671beac9ff204bce85b312bd94870ebf P 560d593b247846858b20f9d7ea174532: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:37.528324 12691 tablet_bootstrap.cc:492] T 671beac9ff204bce85b312bd94870ebf P b584a933f35a49678c7f0adde4e4c211: Bootstrap starting.
I20260806 15:03:37.528483 12687 tablet_bootstrap.cc:492] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5: No bootstrap required, opened a new log
I20260806 15:03:37.528530 12687 ts_tablet_manager.cc:1397] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:03:37.528661 12691 tablet_bootstrap.cc:654] T 671beac9ff204bce85b312bd94870ebf P b584a933f35a49678c7f0adde4e4c211: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:37.528661 12687 raft_consensus.cc:348] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.528719 12687 raft_consensus.cc:374] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:37.528745 12687 raft_consensus.cc:729] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4eb9691b8c3440c925e003fe1a311f5, State: Initialized, Role: FOLLOWER
I20260806 15:03:37.528784 12687 consensus_queue.cc:260] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.528877 12687 ts_tablet_manager.cc:1428] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:37.529287 12691 tablet_bootstrap.cc:492] T 671beac9ff204bce85b312bd94870ebf P b584a933f35a49678c7f0adde4e4c211: No bootstrap required, opened a new log
I20260806 15:03:37.529340 12691 ts_tablet_manager.cc:1397] T 671beac9ff204bce85b312bd94870ebf P b584a933f35a49678c7f0adde4e4c211: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:37.529452 12691 raft_consensus.cc:348] T 671beac9ff204bce85b312bd94870ebf P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.529522 12691 raft_consensus.cc:374] T 671beac9ff204bce85b312bd94870ebf P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:37.529541 12691 raft_consensus.cc:729] T 671beac9ff204bce85b312bd94870ebf P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b584a933f35a49678c7f0adde4e4c211, State: Initialized, Role: FOLLOWER
I20260806 15:03:37.529577 12691 consensus_queue.cc:260] T 671beac9ff204bce85b312bd94870ebf P b584a933f35a49678c7f0adde4e4c211 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.529661 12691 ts_tablet_manager.cc:1428] T 671beac9ff204bce85b312bd94870ebf P b584a933f35a49678c7f0adde4e4c211: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:37.561968 12692 raft_consensus.cc:482] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:03:37.562039 12692 raft_consensus.cc:504] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.562142 12692 leader_election.cc:288] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b584a933f35a49678c7f0adde4e4c211 (127.0.236.65:46041), 560d593b247846858b20f9d7ea174532 (127.0.236.67:45805)
I20260806 15:03:37.562338 12165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "671beac9ff204bce85b312bd94870ebf" candidate_uuid: "c4eb9691b8c3440c925e003fe1a311f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b584a933f35a49678c7f0adde4e4c211" is_pre_election: true
I20260806 15:03:37.562358 12580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "671beac9ff204bce85b312bd94870ebf" candidate_uuid: "c4eb9691b8c3440c925e003fe1a311f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "560d593b247846858b20f9d7ea174532" is_pre_election: true
I20260806 15:03:37.562438 12580 raft_consensus.cc:2454] T 671beac9ff204bce85b312bd94870ebf P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4eb9691b8c3440c925e003fe1a311f5 in term 0.
I20260806 15:03:37.562438 12165 raft_consensus.cc:2454] T 671beac9ff204bce85b312bd94870ebf P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4eb9691b8c3440c925e003fe1a311f5 in term 0.
I20260806 15:03:37.562567 12278 leader_election.cc:302] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 560d593b247846858b20f9d7ea174532, c4eb9691b8c3440c925e003fe1a311f5; no voters:
I20260806 15:03:37.562655 12692 raft_consensus.cc:2781] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:03:37.562708 12692 raft_consensus.cc:482] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:03:37.562728 12692 raft_consensus.cc:3037] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:37.563357 12692 raft_consensus.cc:504] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.563469 12692 leader_election.cc:288] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5 [CANDIDATE]: Term 1 election: Requested vote from peers b584a933f35a49678c7f0adde4e4c211 (127.0.236.65:46041), 560d593b247846858b20f9d7ea174532 (127.0.236.67:45805)
I20260806 15:03:37.563655 12580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "671beac9ff204bce85b312bd94870ebf" candidate_uuid: "c4eb9691b8c3440c925e003fe1a311f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "560d593b247846858b20f9d7ea174532"
I20260806 15:03:37.563653 12165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "671beac9ff204bce85b312bd94870ebf" candidate_uuid: "c4eb9691b8c3440c925e003fe1a311f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b584a933f35a49678c7f0adde4e4c211"
I20260806 15:03:37.563717 12580 raft_consensus.cc:3037] T 671beac9ff204bce85b312bd94870ebf P 560d593b247846858b20f9d7ea174532 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:37.563721 12165 raft_consensus.cc:3037] T 671beac9ff204bce85b312bd94870ebf P b584a933f35a49678c7f0adde4e4c211 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:37.564234 12580 raft_consensus.cc:2454] T 671beac9ff204bce85b312bd94870ebf P 560d593b247846858b20f9d7ea174532 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4eb9691b8c3440c925e003fe1a311f5 in term 1.
I20260806 15:03:37.564240 12165 raft_consensus.cc:2454] T 671beac9ff204bce85b312bd94870ebf P b584a933f35a49678c7f0adde4e4c211 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4eb9691b8c3440c925e003fe1a311f5 in term 1.
I20260806 15:03:37.564381 12278 leader_election.cc:302] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b584a933f35a49678c7f0adde4e4c211, c4eb9691b8c3440c925e003fe1a311f5; no voters:
I20260806 15:03:37.564474 12692 raft_consensus.cc:2781] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:37.564582 12692 raft_consensus.cc:686] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5 [term 1 LEADER]: Becoming Leader. State: Replica: c4eb9691b8c3440c925e003fe1a311f5, State: Running, Role: LEADER
I20260806 15:03:37.564640 12692 consensus_queue.cc:237] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } }
I20260806 15:03:37.565208 11918 catalog_manager.cc:5495] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4eb9691b8c3440c925e003fe1a311f5 (127.0.236.66). New cstate: current_term: 1 leader_uuid: "c4eb9691b8c3440c925e003fe1a311f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4eb9691b8c3440c925e003fe1a311f5" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 44243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 } health_report { overall_health: UNKNOWN } } }
I20260806 15:03:37.577648 12833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:37.575510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52846 (local address 127.0.236.66:44243)
0806 15:03:37.575663 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:37.575665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:37.575672 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:03:37.575704 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:37.575705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:37.575722 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:37.575787 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:03:37.575883 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.576367 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.577058 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.577248 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.577290 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:37.577385 (+ 95us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:03:37.577488 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:03:37.577535 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:37.577550 (+ 15us) server_negotiation.cc:299] Negotiation successful
0806 15:03:37.577565 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":98,"threads_started":1}
I20260806 15:03:37.581735 12684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:37.579802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:34661 (local address 127.0.236.67:45805)
0806 15:03:37.579843 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:37.579845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:37.579930 (+ 85us) server_negotiation.cc:407] Connection header received
0806 15:03:37.579975 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:37.579975 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:37.579982 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:37.580023 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:03:37.580112 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.580561 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.581237 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.581404 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.581448 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:37.581561 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:03:37.581584 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:03:37.581626 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:37.581677 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 15:03:37.581690 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 15:03:37.581769 12836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:37.579746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:45805 (local address 127.0.236.66:34661)
0806 15:03:37.579879 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:37.579889 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:37.579939 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:37.580026 (+ 87us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:37.580027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:37.580028 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 15:03:37.580081 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:37.580082 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.580621 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.580622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:37.581180 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:37.581182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.581450 (+ 268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.581451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:37.581495 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:37.581508 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 15:03:37.581636 (+ 128us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 15:03:37.581637 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 15:03:37.581677 (+ 40us) client_negotiation.cc:240] Negotiation successful
0806 15:03:37.581708 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":59,"threads_started":1}
I20260806 15:03:37.583410 12580 raft_consensus.cc:1261] T 671beac9ff204bce85b312bd94870ebf P 560d593b247846858b20f9d7ea174532 [term 1 FOLLOWER]: Refusing update from remote peer c4eb9691b8c3440c925e003fe1a311f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:37.583453 12165 raft_consensus.cc:1261] T 671beac9ff204bce85b312bd94870ebf P b584a933f35a49678c7f0adde4e4c211 [term 1 FOLLOWER]: Refusing update from remote peer c4eb9691b8c3440c925e003fe1a311f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:37.583627 12831 consensus_queue.cc:1035] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "560d593b247846858b20f9d7ea174532" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 45805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:37.583750 12692 consensus_queue.cc:1035] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b584a933f35a49678c7f0adde4e4c211" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:37.589646 12833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:37.587788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:59783 (local address 127.0.236.66:44243)
0806 15:03:37.587823 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:37.587825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:37.587833 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:37.587875 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:37.587876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:37.587884 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:37.587922 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:03:37.588029 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.588540 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.589167 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.589331 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.589368 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:37.589496 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:03:37.589517 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:03:37.589547 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:37.589578 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 15:03:37.589590 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:03:37.589684 12792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:37.587745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:44243 (local address 127.0.236.67:59783)
0806 15:03:37.587784 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:37.587793 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:37.587831 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:37.587942 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:37.587942 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:37.587944 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 15:03:37.587979 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:37.587980 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.588598 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.588599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:37.589110 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:37.589111 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:37.589383 (+ 272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:37.589383 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:37.589435 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:37.589441 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 15:03:37.589548 (+ 107us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 15:03:37.589549 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 15:03:37.589577 (+ 28us) client_negotiation.cc:240] Negotiation successful
0806 15:03:37.589626 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260806 15:03:37.596410 11918 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:37.596491 11918 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:37.597029 11918 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 366abdf501094dc593ca43cdbfc488ef: Sending DeleteTablet for 3 replicas of tablet 671beac9ff204bce85b312bd94870ebf
I20260806 15:03:37.597257 12134 tablet_service.cc:1514] Processing DeleteTablet for tablet 671beac9ff204bce85b312bd94870ebf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:37 UTC) from {username='oryx', principal='oryx/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:39000
I20260806 15:03:37.597275 12553 tablet_service.cc:1514] Processing DeleteTablet for tablet 671beac9ff204bce85b312bd94870ebf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:37 UTC) from {username='oryx', principal='oryx/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:55312
I20260806 15:03:37.597344 12734 tablet_replica.cc:330] T 671beac9ff204bce85b312bd94870ebf P b584a933f35a49678c7f0adde4e4c211: stopping tablet replica
I20260806 15:03:37.597355 12729 tablet_replica.cc:330] T 671beac9ff204bce85b312bd94870ebf P 560d593b247846858b20f9d7ea174532: stopping tablet replica
I20260806 15:03:37.597394 12734 raft_consensus.cc:2229] T 671beac9ff204bce85b312bd94870ebf P b584a933f35a49678c7f0adde4e4c211 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:37.597398 12729 raft_consensus.cc:2229] T 671beac9ff204bce85b312bd94870ebf P 560d593b247846858b20f9d7ea174532 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:37.597429 12729 raft_consensus.cc:2258] T 671beac9ff204bce85b312bd94870ebf P 560d593b247846858b20f9d7ea174532 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:37.597429 12734 raft_consensus.cc:2258] T 671beac9ff204bce85b312bd94870ebf P b584a933f35a49678c7f0adde4e4c211 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:37.597505 12344 tablet_service.cc:1514] Processing DeleteTablet for tablet 671beac9ff204bce85b312bd94870ebf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:37 UTC) from {username='oryx', principal='oryx/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:52844
I20260806 15:03:37.597579 12728 tablet_replica.cc:330] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5: stopping tablet replica
I20260806 15:03:37.597618 12728 raft_consensus.cc:2229] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:03:37.597672 12728 raft_consensus.cc:2258] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:37.597805 12729 ts_tablet_manager.cc:1905] T 671beac9ff204bce85b312bd94870ebf P 560d593b247846858b20f9d7ea174532: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:37.598033 12734 ts_tablet_manager.cc:1905] T 671beac9ff204bce85b312bd94870ebf P b584a933f35a49678c7f0adde4e4c211: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:37.599206 12734 ts_tablet_manager.cc:1918] T 671beac9ff204bce85b312bd94870ebf P b584a933f35a49678c7f0adde4e4c211: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:03:37.599257 12734 log.cc:1196] T 671beac9ff204bce85b312bd94870ebf P b584a933f35a49678c7f0adde4e4c211: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-0/wal/wals/671beac9ff204bce85b312bd94870ebf
I20260806 15:03:37.599270 12729 ts_tablet_manager.cc:1918] T 671beac9ff204bce85b312bd94870ebf P 560d593b247846858b20f9d7ea174532: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:03:37.599329 12729 log.cc:1196] T 671beac9ff204bce85b312bd94870ebf P 560d593b247846858b20f9d7ea174532: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/671beac9ff204bce85b312bd94870ebf
I20260806 15:03:37.599457 12734 ts_tablet_manager.cc:1939] T 671beac9ff204bce85b312bd94870ebf P b584a933f35a49678c7f0adde4e4c211: Deleting consensus metadata
I20260806 15:03:37.599540 12729 ts_tablet_manager.cc:1939] T 671beac9ff204bce85b312bd94870ebf P 560d593b247846858b20f9d7ea174532: Deleting consensus metadata
I20260806 15:03:37.599793 12728 ts_tablet_manager.cc:1905] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:37.599781 11895 catalog_manager.cc:4841] TS b584a933f35a49678c7f0adde4e4c211 (127.0.236.65:46041): tablet 671beac9ff204bce85b312bd94870ebf (table test-table [id=485431eba42242febdee23382cd14d86]) successfully deleted
I20260806 15:03:37.599854 11895 catalog_manager.cc:4841] TS 560d593b247846858b20f9d7ea174532 (127.0.236.67:45805): tablet 671beac9ff204bce85b312bd94870ebf (table test-table [id=485431eba42242febdee23382cd14d86]) successfully deleted
I20260806 15:03:37.600838 12728 ts_tablet_manager.cc:1918] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:03:37.600888 12728 log.cc:1196] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028580791223-945-0/minicluster-data/ts-1/wal/wals/671beac9ff204bce85b312bd94870ebf
I20260806 15:03:37.601050 12728 ts_tablet_manager.cc:1939] T 671beac9ff204bce85b312bd94870ebf P c4eb9691b8c3440c925e003fe1a311f5: Deleting consensus metadata
I20260806 15:03:37.601334 11899 catalog_manager.cc:4841] TS c4eb9691b8c3440c925e003fe1a311f5 (127.0.236.66:44243): tablet 671beac9ff204bce85b312bd94870ebf (table test-table [id=485431eba42242febdee23382cd14d86]) successfully deleted
I20260806 15:03:37.601567 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 12029
I20260806 15:03:37.618055 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 12255
I20260806 15:03:37.630023 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 12458
I20260806 15:03:37.640309 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 11868
2026-08-06T15:03:37Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3097 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:03:37 dist-test-slave-tc2s krb5kdc[12849](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:03:37 dist-test-slave-tc2s krb5kdc[12849](info): set up 2 sockets
Aug 06 15:03:37 dist-test-slave-tc2s krb5kdc[12849](info): commencing operation
krb5kdc: starting...
W20260806 15:03:38.751538 945 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:03:38 dist-test-slave-tc2s krb5kdc[12849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:03:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:03:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:03:38.857851 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:37511
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:40031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:37511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:38.936825 12867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:38.937003 12867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:38.937031 12867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:38.938148 12867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:03:38.938185 12867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:38.938198 12867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:38.938220 12867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:03:38.938243 12867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:03:38.939692 12867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:40031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:37511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:37511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:38.939903 12867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:38.940099 12867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:38.944582 12875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:38.944593 12877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:38.944633 12873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:38.944738 12867 server_base.cc:1034] running on GCE node
I20260806 15:03:38.944979 12867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:38.945243 12867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:38.946370 12867 hybrid_clock.cc:648] HybridClock initialized: now 1786028618946358 us; error 27 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T15:03:38Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1311 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:03:39 dist-test-slave-tc2s krb5kdc[12884](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:03:39 dist-test-slave-tc2s krb5kdc[12884](info): set up 2 sockets
Aug 06 15:03:39 dist-test-slave-tc2s krb5kdc[12884](info): commencing operation
krb5kdc: starting...
W20260806 15:03:40.058169 945 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:03:40 dist-test-slave-tc2s krb5kdc[12884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:03:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:03:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:03:40.165683 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:43829
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:34977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:43829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:40.243310 12905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:40.243500 12905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:40.243530 12905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:40.244625 12905 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:03:40.244663 12905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:40.244683 12905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:40.244697 12905 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:03:40.244712 12905 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:03:40.246243 12905 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:34977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:43829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:43829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:40.246479 12905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:40.246677 12905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:40.251183 12914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:40.251196 12913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:40.251266 12905 server_base.cc:1034] running on GCE node
W20260806 15:03:40.251196 12917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:40.251633 12905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:40.251905 12905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:40.253046 12905 hybrid_clock.cc:648] HybridClock initialized: now 1786028620253032 us; error 30 us; skew 500 ppm
Aug 06 15:03:40 dist-test-slave-tc2s krb5kdc[12884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028620, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:40.258313 12905 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:03:40.258709 12905 webserver.cc:460] Webserver started at http://127.0.236.126:46045/ using document root <none> and password file <none>
I20260806 15:03:40.258904 12905 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:40.258968 12905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:40.259090 12905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:40.260057 12905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "e5af8a8e298b4f119c7218fe9b2d08c3"
format_stamp: "Formatted at 2026-08-06 15:03:40 on dist-test-slave-tc2s"
server_key: "3b533a518729d8b32add33d563be4584"
server_key_iv: "912c026baaf53c3c7a0aa62f7648b8ee"
server_key_version: "encryptionkey@0"
I20260806 15:03:40.260381 12905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "e5af8a8e298b4f119c7218fe9b2d08c3"
format_stamp: "Formatted at 2026-08-06 15:03:40 on dist-test-slave-tc2s"
server_key: "3b533a518729d8b32add33d563be4584"
server_key_iv: "912c026baaf53c3c7a0aa62f7648b8ee"
server_key_version: "encryptionkey@0"
I20260806 15:03:40.261567 12905 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 15:03:40.262228 12929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:40.262373 12905 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:40.262462 12905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "e5af8a8e298b4f119c7218fe9b2d08c3"
format_stamp: "Formatted at 2026-08-06 15:03:40 on dist-test-slave-tc2s"
server_key: "3b533a518729d8b32add33d563be4584"
server_key_iv: "912c026baaf53c3c7a0aa62f7648b8ee"
server_key_version: "encryptionkey@0"
I20260806 15:03:40.262521 12905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:40.279987 12905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:40.281579 12905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:40.281697 12905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:40.285399 12905 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:43829
I20260806 15:03:40.285456 13007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:43829 every 8 connection(s)
I20260806 15:03:40.285879 12905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:03:40.286521 13008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:40.288604 13008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3: Bootstrap starting.
I20260806 15:03:40.289127 13008 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:40.289431 13008 log.cc:824] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:40.289455 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 12905
I20260806 15:03:40.289553 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:03:40.289741 945 external_mini_cluster.cc:1428] Setting key 1179107bad03f29900f719ff49946fae
I20260806 15:03:40.290189 13008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3: No bootstrap required, opened a new log
I20260806 15:03:40.292702 13008 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5af8a8e298b4f119c7218fe9b2d08c3" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43829 } }
I20260806 15:03:40.292860 13008 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:40.292891 13008 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5af8a8e298b4f119c7218fe9b2d08c3, State: Initialized, Role: FOLLOWER
I20260806 15:03:40.293015 13008 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5af8a8e298b4f119c7218fe9b2d08c3" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43829 } }
I20260806 15:03:40.293085 13008 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:03:40.293118 13008 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:03:40.293159 13008 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:40.293764 13008 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5af8a8e298b4f119c7218fe9b2d08c3" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43829 } }
I20260806 15:03:40.293851 13008 leader_election.cc:302] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5af8a8e298b4f119c7218fe9b2d08c3; no voters:
I20260806 15:03:40.294016 13008 leader_election.cc:288] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:03:40.294075 13019 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:40.294225 13019 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [term 1 LEADER]: Becoming Leader. State: Replica: e5af8a8e298b4f119c7218fe9b2d08c3, State: Running, Role: LEADER
I20260806 15:03:40.294242 13008 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:03:40.294339 13019 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5af8a8e298b4f119c7218fe9b2d08c3" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43829 } }
I20260806 15:03:40.294687 13020 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5af8a8e298b4f119c7218fe9b2d08c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5af8a8e298b4f119c7218fe9b2d08c3" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43829 } } }
I20260806 15:03:40.294765 13020 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:40.295159 13021 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5af8a8e298b4f119c7218fe9b2d08c3. Latest consensus state: current_term: 1 leader_uuid: "e5af8a8e298b4f119c7218fe9b2d08c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5af8a8e298b4f119c7218fe9b2d08c3" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43829 } } }
I20260806 15:03:40.295233 13021 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:40.295852 13026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:03:40.296362 13026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 15:03:40 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:40.297920 13026 catalog_manager.cc:1347] Generated new cluster ID: 28f3c4d917b840c1aa8e6979a9ba1982
I20260806 15:03:40.297976 13026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:03:40.301950 13017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:40.290213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55242 (local address 127.0.236.126:43829)
0806 15:03:40.290394 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:40.290403 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:40.290445 (+ 42us) server_negotiation.cc:407] Connection header received
0806 15:03:40.291390 (+ 945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:40.291399 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:40.291475 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:40.291619 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:40.292186 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:40.293518 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:40.294885 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:40.295581 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:40.295669 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:40.298284 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:40.298287 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:40.298288 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:40.298298 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:40.300062 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:40.300469 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:40.300472 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:40.300472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:40.300568 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:40.300761 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:40.300761 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:40.300761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:40.301023 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:40.301159 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:40.301281 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 15:03:40.301407 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260806 15:03:40.302697 13026 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:03:40.303196 13026 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:03:40.307837 13026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3: Generated new TSK 0
I20260806 15:03:40.307988 13026 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.236.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
I20260806 15:03:40.330327 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:0
--local_ip_for_outbound_sockets=127.0.236.65
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:43829
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:34977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:40.408625 13045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:40.408815 13045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:40.408838 13045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:40.409958 13045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:40.409997 13045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:40.410022 13045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:03:40.411495 13045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:34977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:43829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:40.411706 13045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:40.411911 13045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:40.416585 13051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:40.416625 13050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:40.416587 13054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:40.416841 13045 server_base.cc:1034] running on GCE node
I20260806 15:03:40.417026 13045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:40.417245 13045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:40.418387 13045 hybrid_clock.cc:648] HybridClock initialized: now 1786028620418373 us; error 28 us; skew 500 ppm
Aug 06 15:03:40 dist-test-slave-tc2s krb5kdc[12884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028620, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:40.420648 13045 init.cc:377] Logged in from keytab as kudu/127.0.236.65@KRBTEST.COM (short username kudu)
I20260806 15:03:40.421033 13045 webserver.cc:460] Webserver started at http://127.0.236.65:36253/ using document root <none> and password file <none>
I20260806 15:03:40.421226 13045 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:40.421272 13045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:40.421396 13045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:40.422407 13045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/data/instance:
uuid: "51792dfe11644c059c02df1f0b2684c3"
format_stamp: "Formatted at 2026-08-06 15:03:40 on dist-test-slave-tc2s"
server_key: "0ce8a64a79aaff2791164f7af80cde80"
server_key_iv: "02e2e67b643f17cd7da069a50ffa0a08"
server_key_version: "encryptionkey@0"
I20260806 15:03:40.422720 13045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/wal/instance:
uuid: "51792dfe11644c059c02df1f0b2684c3"
format_stamp: "Formatted at 2026-08-06 15:03:40 on dist-test-slave-tc2s"
server_key: "0ce8a64a79aaff2791164f7af80cde80"
server_key_iv: "02e2e67b643f17cd7da069a50ffa0a08"
server_key_version: "encryptionkey@0"
I20260806 15:03:40.423902 13045 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:03:40.424526 13064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:40.424669 13045 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:40.424747 13045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "51792dfe11644c059c02df1f0b2684c3"
format_stamp: "Formatted at 2026-08-06 15:03:40 on dist-test-slave-tc2s"
server_key: "0ce8a64a79aaff2791164f7af80cde80"
server_key_iv: "02e2e67b643f17cd7da069a50ffa0a08"
server_key_version: "encryptionkey@0"
I20260806 15:03:40.424803 13045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:40.442605 13045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:40.444124 13045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:40.444245 13045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:40.444433 13045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:40.444708 13045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:40.444747 13045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:40.444775 13045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:40.444798 13045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:40.450131 13045 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:34981
I20260806 15:03:40.450170 13234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:34981 every 8 connection(s)
I20260806 15:03:40.450438 13045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:03:40.454489 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 13045
I20260806 15:03:40.454614 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/wal/instance
I20260806 15:03:40.454806 945 external_mini_cluster.cc:1428] Setting key 26c28c605380d50dbb3c6550d226f4aa
Aug 06 15:03:40 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028620, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:40.458874 13017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:40.451035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:59515 (local address 127.0.236.126:43829)
0806 15:03:40.451109 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:40.451111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:40.451263 (+ 152us) server_negotiation.cc:407] Connection header received
0806 15:03:40.452076 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:40.452077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:40.452086 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:40.452132 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:40.452623 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:40.453108 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:40.454019 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:40.454218 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:40.454265 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:40.456367 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:40.456368 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:40.456369 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:40.456377 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:40.457617 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:40.458028 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:40.458028 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:40.458029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:40.458081 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:40.458295 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:40.458295 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:40.458296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:40.458401 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:40.458467 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:40.458705 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 15:03:40.458788 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 15:03:40.459025 13239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:40.451012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.65:59515)
0806 15:03:40.451160 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:40.451192 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:40.451998 (+ 806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:40.452180 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:40.452183 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:40.452397 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:40.452564 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:40.452568 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:40.453165 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:40.453166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:40.453959 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:40.453961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:40.454271 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:40.454272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:40.454362 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:40.454814 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:40.454822 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:40.456264 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:40.457693 (+ 1429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:40.457700 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:40.457702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:40.457951 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:40.458133 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:40.458134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:40.458135 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:40.458238 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:40.458459 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:40.458465 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:40.458594 (+ 129us) client_negotiation.cc:769] Sending connection context
0806 15:03:40.458678 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 15:03:40.458817 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
I20260806 15:03:40.459447 13235 heartbeater.cc:347] Connected to a master server at 127.0.236.126:43829
I20260806 15:03:40.459547 13235 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:40.459719 13235 heartbeater.cc:511] Master 127.0.236.126:43829 requested a full tablet report, sending...
I20260806 15:03:40.460187 12954 ts_manager.cc:194] Registered new tserver with Master: 51792dfe11644c059c02df1f0b2684c3 (127.0.236.65:34981)
I20260806 15:03:40.460775 12954 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.65@KRBTEST.COM'} at 127.0.236.65:59515
WARNING: no policy specified for kudu/127.0.236.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
I20260806 15:03:40.477684 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:0
--local_ip_for_outbound_sockets=127.0.236.66
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:43829
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:34977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:40.555444 13243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:40.555622 13243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:40.555644 13243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:40.557065 13243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:40.557106 13243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:40.557130 13243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:03:40.558629 13243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:34977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:43829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:40.558842 13243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:40.559034 13243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:40.563515 13254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:40.563656 13243 server_base.cc:1034] running on GCE node
W20260806 15:03:40.563509 13249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:40.563509 13251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:40.563911 13243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:40.564126 13243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:40.565259 13243 hybrid_clock.cc:648] HybridClock initialized: now 1786028620565243 us; error 29 us; skew 500 ppm
Aug 06 15:03:40 dist-test-slave-tc2s krb5kdc[12884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028620, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:40.567507 13243 init.cc:377] Logged in from keytab as kudu/127.0.236.66@KRBTEST.COM (short username kudu)
I20260806 15:03:40.567864 13243 webserver.cc:460] Webserver started at http://127.0.236.66:45977/ using document root <none> and password file <none>
I20260806 15:03:40.568058 13243 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:40.568102 13243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:40.568220 13243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:40.569125 13243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/data/instance:
uuid: "92a6bd7720ac4495b944e844f9ba5966"
format_stamp: "Formatted at 2026-08-06 15:03:40 on dist-test-slave-tc2s"
server_key: "8ad475949645517523eb9eb5d5e8b06f"
server_key_iv: "2d5e9b688683fbc56ddb4437759b87a3"
server_key_version: "encryptionkey@0"
I20260806 15:03:40.569435 13243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/wal/instance:
uuid: "92a6bd7720ac4495b944e844f9ba5966"
format_stamp: "Formatted at 2026-08-06 15:03:40 on dist-test-slave-tc2s"
server_key: "8ad475949645517523eb9eb5d5e8b06f"
server_key_iv: "2d5e9b688683fbc56ddb4437759b87a3"
server_key_version: "encryptionkey@0"
I20260806 15:03:40.570621 13243 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:03:40.571244 13266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:40.571393 13243 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:40.571478 13243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "92a6bd7720ac4495b944e844f9ba5966"
format_stamp: "Formatted at 2026-08-06 15:03:40 on dist-test-slave-tc2s"
server_key: "8ad475949645517523eb9eb5d5e8b06f"
server_key_iv: "2d5e9b688683fbc56ddb4437759b87a3"
server_key_version: "encryptionkey@0"
I20260806 15:03:40.571542 13243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:40.584823 13243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:40.586527 13243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:40.586644 13243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:40.586820 13243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:40.587085 13243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:40.587116 13243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:40.587136 13243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:40.587158 13243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:40.592446 13243 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:40017
I20260806 15:03:40.592501 13435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:40017 every 8 connection(s)
I20260806 15:03:40.592752 13243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
Aug 06 15:03:40 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028620, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:40.601085 13017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:40.593355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:57739 (local address 127.0.236.126:43829)
0806 15:03:40.593412 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:40.593414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:40.593572 (+ 158us) server_negotiation.cc:407] Connection header received
0806 15:03:40.594512 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:40.594514 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:40.594524 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:40.594572 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:40.595129 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:40.595635 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:40.596531 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:40.596709 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:40.596754 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:40.598659 (+ 1905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:40.598660 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:40.598661 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:40.598668 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:40.599860 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:40.600218 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:40.600219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:40.600219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:40.600273 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:40.600501 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:40.600502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:40.600502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:40.600607 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:40.600662 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:40.600910 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 15:03:40.600998 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 15:03:40.601212 13444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:40.593312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.66:57739)
0806 15:03:40.593465 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:40.593503 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:40.594429 (+ 926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:40.594619 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:40.594622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:40.594883 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:40.595067 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:40.595071 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:40.595695 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:40.595696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:40.596474 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:40.596476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:40.596761 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:40.596762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:40.596849 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:40.597244 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:40.597251 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:40.598551 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:40.599936 (+ 1385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:40.599942 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:40.599944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:40.600152 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:40.600338 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:40.600339 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:40.600339 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:40.600444 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:40.600664 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:40.600670 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:40.600794 (+ 124us) client_negotiation.cc:769] Sending connection context
0806 15:03:40.600876 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 15:03:40.601020 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260806 15:03:40.601650 13439 heartbeater.cc:347] Connected to a master server at 127.0.236.126:43829
I20260806 15:03:40.601737 13439 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:40.601742 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 13243
I20260806 15:03:40.601837 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/wal/instance
I20260806 15:03:40.601922 13439 heartbeater.cc:511] Master 127.0.236.126:43829 requested a full tablet report, sending...
I20260806 15:03:40.602046 945 external_mini_cluster.cc:1428] Setting key a0fe5fbebc6f7b5f09c1b49fffc29a45
I20260806 15:03:40.602296 12954 ts_manager.cc:194] Registered new tserver with Master: 92a6bd7720ac4495b944e844f9ba5966 (127.0.236.66:40017)
I20260806 15:03:40.602795 12954 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.66@KRBTEST.COM'} at 127.0.236.66:57739
WARNING: no policy specified for kudu/127.0.236.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
I20260806 15:03:40.624512 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:0
--local_ip_for_outbound_sockets=127.0.236.67
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:43829
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:34977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:40.703055 13447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:40.703235 13447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:40.703255 13447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:40.704327 13447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:40.704360 13447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:40.704383 13447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:03:40.705793 13447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:34977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:43829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:40.706004 13447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:40.706175 13447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:40.710601 13455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:40.710615 13453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:40.710613 13457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:40.710819 13447 server_base.cc:1034] running on GCE node
I20260806 15:03:40.711036 13447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:40.711249 13447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:40.712385 13447 hybrid_clock.cc:648] HybridClock initialized: now 1786028620712371 us; error 30 us; skew 500 ppm
Aug 06 15:03:40 dist-test-slave-tc2s krb5kdc[12884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028620, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:40.714716 13447 init.cc:377] Logged in from keytab as kudu/127.0.236.67@KRBTEST.COM (short username kudu)
I20260806 15:03:40.715132 13447 webserver.cc:460] Webserver started at http://127.0.236.67:42287/ using document root <none> and password file <none>
I20260806 15:03:40.715317 13447 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:40.715363 13447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:40.715484 13447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:40.716403 13447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/data/instance:
uuid: "642a40d30ca3499ebb247283efc01d91"
format_stamp: "Formatted at 2026-08-06 15:03:40 on dist-test-slave-tc2s"
server_key: "9aa47bac671fb3af54808d19f6934f59"
server_key_iv: "988ae1d8f0391aca46163930b4bb0a37"
server_key_version: "encryptionkey@0"
I20260806 15:03:40.716710 13447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/wal/instance:
uuid: "642a40d30ca3499ebb247283efc01d91"
format_stamp: "Formatted at 2026-08-06 15:03:40 on dist-test-slave-tc2s"
server_key: "9aa47bac671fb3af54808d19f6934f59"
server_key_iv: "988ae1d8f0391aca46163930b4bb0a37"
server_key_version: "encryptionkey@0"
I20260806 15:03:40.717861 13447 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 15:03:40.718567 13467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:40.718712 13447 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:40.718793 13447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "642a40d30ca3499ebb247283efc01d91"
format_stamp: "Formatted at 2026-08-06 15:03:40 on dist-test-slave-tc2s"
server_key: "9aa47bac671fb3af54808d19f6934f59"
server_key_iv: "988ae1d8f0391aca46163930b4bb0a37"
server_key_version: "encryptionkey@0"
I20260806 15:03:40.718856 13447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:40.747414 13447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:40.749465 13447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:40.749601 13447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:40.749814 13447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:40.750144 13447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:40.750222 13447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:40.750264 13447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:40.750281 13447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:40.755532 13447 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:38069
I20260806 15:03:40.755579 13631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:38069 every 8 connection(s)
I20260806 15:03:40.755873 13447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
I20260806 15:03:40.758720 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 13447
I20260806 15:03:40.758823 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/wal/instance
I20260806 15:03:40.758993 945 external_mini_cluster.cc:1428] Setting key b08e51864d3599857eaaa733dcb96573
Aug 06 15:03:40 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028620, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:40.764570 13017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:40.756560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:58111 (local address 127.0.236.126:43829)
0806 15:03:40.756620 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:40.756623 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:40.756781 (+ 158us) server_negotiation.cc:407] Connection header received
0806 15:03:40.757638 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:40.757639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:40.757650 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:40.757690 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:40.758268 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:40.758754 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:40.759983 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:40.760178 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:40.760234 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:40.762165 (+ 1931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:40.762167 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:40.762167 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:40.762175 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:40.763368 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:40.763728 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:40.763730 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:40.763730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:40.763783 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:40.763994 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:40.763995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:40.763996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:40.764102 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:40.764156 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:40.764401 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 15:03:40.764482 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 15:03:40.764743 13635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:40.756526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.67:58111)
0806 15:03:40.756678 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:40.756711 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:40.757547 (+ 836us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:40.757752 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:40.757755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:40.758030 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:40.758208 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:40.758212 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:40.758840 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:40.758841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:40.759908 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:40.759911 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:40.760266 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:40.760268 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:40.760395 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:40.760803 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:40.760811 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:40.762083 (+ 1272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:40.763451 (+ 1368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:40.763458 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:40.763460 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:40.763663 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:40.763835 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:40.763836 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:40.763836 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:40.763940 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:40.764156 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:40.764163 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:40.764288 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 15:03:40.764372 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 15:03:40.764547 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":77,"threads_started":1}
I20260806 15:03:40.765111 13633 heartbeater.cc:347] Connected to a master server at 127.0.236.126:43829
I20260806 15:03:40.765197 13633 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:40.765373 13633 heartbeater.cc:511] Master 127.0.236.126:43829 requested a full tablet report, sending...
I20260806 15:03:40.765729 12954 ts_manager.cc:194] Registered new tserver with Master: 642a40d30ca3499ebb247283efc01d91 (127.0.236.67:38069)
I20260806 15:03:40.766121 12954 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.67@KRBTEST.COM'} at 127.0.236.67:58111
I20260806 15:03:40.770462 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.0.236.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.236.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.236.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/oryx_127.0.236.65.keytab.
Entry for principal oryx/127.0.236.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/oryx_127.0.236.65.keytab.
WARNING: no policy specified for oryx/127.0.236.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.236.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.236.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/oryx_127.0.236.66.keytab.
Entry for principal oryx/127.0.236.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/oryx_127.0.236.66.keytab.
WARNING: no policy specified for oryx/127.0.236.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.236.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.236.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/oryx_127.0.236.67.keytab.
Entry for principal oryx/127.0.236.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/oryx_127.0.236.67.keytab.
I20260806 15:03:40.832047 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 13045
I20260806 15:03:40.844010 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 13243
I20260806 15:03:40.849805 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 13447
I20260806 15:03:40.858372 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 12905
I20260806 15:03:40.867831 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:43829
--webserver_interface=127.0.236.126
--webserver_port=46045
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:34977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:43829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:40.951193 13642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:40.951397 13642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:40.951418 13642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:40.952495 13642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:03:40.952526 13642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:40.952538 13642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:40.952549 13642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:03:40.952566 13642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:03:40.954057 13642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:34977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:43829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:43829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=46045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:40.954252 13642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:40.954421 13642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:40.958745 13649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:40.958839 13642 server_base.cc:1034] running on GCE node
W20260806 15:03:40.958877 13653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:40.958755 13647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:40.959167 13642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:40.959406 13642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:40.960786 13642 hybrid_clock.cc:648] HybridClock initialized: now 1786028620960768 us; error 33 us; skew 500 ppm
Aug 06 15:03:40 dist-test-slave-tc2s krb5kdc[12884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028620, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:40.963212 13642 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:03:40.963624 13642 webserver.cc:460] Webserver started at http://127.0.236.126:46045/ using document root <none> and password file <none>
I20260806 15:03:40.963825 13642 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:40.963888 13642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:40.965122 13642 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:40.965684 13660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:40.965843 13642 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:40.965945 13642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "e5af8a8e298b4f119c7218fe9b2d08c3"
format_stamp: "Formatted at 2026-08-06 15:03:40 on dist-test-slave-tc2s"
server_key: "3b533a518729d8b32add33d563be4584"
server_key_iv: "912c026baaf53c3c7a0aa62f7648b8ee"
server_key_version: "encryptionkey@0"
I20260806 15:03:40.966243 13642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:40.975973 13642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:40.977476 13642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:40.977595 13642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:40.981292 13642 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:43829
I20260806 15:03:40.981345 13734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:43829 every 8 connection(s)
I20260806 15:03:40.981803 13642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:03:40.982301 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 13642
I20260806 15:03:40.982566 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:34981
--local_ip_for_outbound_sockets=127.0.236.65
--tserver_master_addrs=127.0.236.126:43829
--webserver_port=36253
--webserver_interface=127.0.236.65
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:34977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.236.65
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/oryx_127.0.236.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:03:40.983114 13735 sys_catalog.cc:263] Verifying existing consensus state
I20260806 15:03:40.983732 13735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3: Bootstrap starting.
I20260806 15:03:40.992033 13735 log.cc:824] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:40.993605 13735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 15:03:40.993853 13735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3: Bootstrap complete.
I20260806 15:03:40.995867 13735 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5af8a8e298b4f119c7218fe9b2d08c3" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43829 } }
I20260806 15:03:40.996075 13735 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5af8a8e298b4f119c7218fe9b2d08c3, State: Initialized, Role: FOLLOWER
I20260806 15:03:40.996228 13735 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5af8a8e298b4f119c7218fe9b2d08c3" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43829 } }
I20260806 15:03:40.996289 13735 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:03:40.996328 13735 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:03:40.996373 13735 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [term 1 FOLLOWER]: Advancing to term 2
I20260806 15:03:40.997303 13735 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5af8a8e298b4f119c7218fe9b2d08c3" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43829 } }
I20260806 15:03:40.997390 13735 leader_election.cc:302] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5af8a8e298b4f119c7218fe9b2d08c3; no voters:
I20260806 15:03:40.997560 13735 leader_election.cc:288] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 15:03:40.997612 13742 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 15:03:40.997727 13742 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [term 2 LEADER]: Becoming Leader. State: Replica: e5af8a8e298b4f119c7218fe9b2d08c3, State: Running, Role: LEADER
I20260806 15:03:40.997821 13735 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:03:40.997820 13742 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5af8a8e298b4f119c7218fe9b2d08c3" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43829 } }
I20260806 15:03:40.998422 13744 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5af8a8e298b4f119c7218fe9b2d08c3. Latest consensus state: current_term: 2 leader_uuid: "e5af8a8e298b4f119c7218fe9b2d08c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5af8a8e298b4f119c7218fe9b2d08c3" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43829 } } }
I20260806 15:03:40.998502 13744 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:40.998639 13745 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e5af8a8e298b4f119c7218fe9b2d08c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5af8a8e298b4f119c7218fe9b2d08c3" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43829 } } }
I20260806 15:03:40.998692 13745 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:40.998796 13753 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:03:40.999276 13753 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:03:40.999416 13753 catalog_manager.cc:1259] Loaded cluster ID: 28f3c4d917b840c1aa8e6979a9ba1982
I20260806 15:03:40.999606 13753 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:03:41.000885 13753 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:03:41.001065 13753 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e5af8a8e298b4f119c7218fe9b2d08c3: Loaded TSK: 0
I20260806 15:03:41.001271 13753 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 15:03:41.064965 13739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:41.065145 13739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:41.065166 13739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:41.066322 13739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:41.066361 13739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:41.066388 13739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:03:41.067799 13739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:34977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/oryx_127.0.236.65.keytab
--principal=oryx/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:34981
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=36253
--tserver_master_addrs=127.0.236.126:43829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:41.068017 13739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:41.068219 13739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:41.072836 13771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:41.072844 13776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:41.073017 13773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:41.073247 13739 server_base.cc:1034] running on GCE node
I20260806 15:03:41.073421 13739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:41.073644 13739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:41.074791 13739 hybrid_clock.cc:648] HybridClock initialized: now 1786028621074775 us; error 34 us; skew 500 ppm
Aug 06 15:03:41 dist-test-slave-tc2s krb5kdc[12884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028621, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:41.077478 13739 init.cc:377] Logged in from keytab as oryx/127.0.236.65@KRBTEST.COM (short username oryx)
I20260806 15:03:41.078048 13739 webserver.cc:460] Webserver started at http://127.0.236.65:36253/ using document root <none> and password file <none>
I20260806 15:03:41.078325 13739 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:41.078408 13739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:41.080116 13739 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:03:41.080803 13791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:41.080976 13739 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:41.081077 13739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "51792dfe11644c059c02df1f0b2684c3"
format_stamp: "Formatted at 2026-08-06 15:03:40 on dist-test-slave-tc2s"
server_key: "0ce8a64a79aaff2791164f7af80cde80"
server_key_iv: "02e2e67b643f17cd7da069a50ffa0a08"
server_key_version: "encryptionkey@0"
I20260806 15:03:41.081496 13739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:41.101590 13739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:41.103842 13739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:41.104000 13739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:41.104228 13739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:41.104548 13739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:41.104589 13739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:41.104624 13739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:41.104647 13739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:41.111779 13739 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:34981
I20260806 15:03:41.111812 13951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:34981 every 8 connection(s)
I20260806 15:03:41.112267 13739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:03:41.118119 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 13739
I20260806 15:03:41.118417 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:40017
--local_ip_for_outbound_sockets=127.0.236.66
--tserver_master_addrs=127.0.236.126:43829
--webserver_port=45977
--webserver_interface=127.0.236.66
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:34977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.236.66
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/oryx_127.0.236.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 15:03:41 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:41 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:41.120178 13954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:41.112954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.65:53323)
0806 15:03:41.113134 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:41.113205 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:41.113680 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:41.114129 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:41.114133 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:41.114674 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:41.114847 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:41.114851 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.115919 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.115921 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:41.116689 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:41.116692 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.117263 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.117265 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:41.117393 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:41.117824 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:41.117832 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:41.119995 (+ 2163us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":90,"threads_started":1}
W20260806 15:03:41.120473 13955 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:41.112990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:53323 (local address 127.0.236.126:43829)
0806 15:03:41.113179 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:41.113188 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:41.113299 (+ 111us) server_negotiation.cc:407] Connection header received
0806 15:03:41.113877 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:41.113881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:41.113967 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:41.114082 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:41.114968 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.115815 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.116774 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.117186 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.117277 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:41.120281 (+ 3004us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.65:53323: BlockingRecv error: recv got EOF from 127.0.236.65:53323 (error 108)
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":84,"threads_started":1}
W20260806 15:03:41.120615 13952 heartbeater.cc:644] Failed to heartbeat to 127.0.236.126:43829 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.236.126:43829: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Aug 06 15:03:41 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:41 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:41.125982 13954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:41.120831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.65:56357)
0806 15:03:41.120968 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:41.120977 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:41.121024 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:41.121161 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:41.121162 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:41.121300 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:41.121347 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:41.121348 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.122264 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.122265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:41.123015 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:41.123018 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.123455 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.123456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:41.123523 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:41.123920 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:41.123925 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:41.125908 (+ 1983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":81,"threads_started":1}
W20260806 15:03:41.126107 13955 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:41.120865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:56357 (local address 127.0.236.126:43829)
0806 15:03:41.120922 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:41.120924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:41.121015 (+ 91us) server_negotiation.cc:407] Connection header received
0806 15:03:41.121085 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:41.121086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:41.121099 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:41.121148 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:41.121420 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.122174 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.123105 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.123374 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.123448 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:41.126047 (+ 2599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.65:56357: BlockingRecv error: recv got EOF from 127.0.236.65:56357 (error 108)
Metrics: {"mutex_wait_us":12,"server-negotiator.queue_time_us":39}
Aug 06 15:03:41 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:41 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:41.131111 13954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:41.126432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.65:49073)
0806 15:03:41.126500 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:41.126509 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:41.126555 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:41.126678 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:41.126679 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:41.126816 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:41.126849 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:41.126850 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.127699 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.127700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:41.128435 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:41.128438 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.128873 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.128874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:41.128927 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:41.129283 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:41.129288 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:41.131050 (+ 1762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 15:03:41.131198 13955 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:41.126482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:49073 (local address 127.0.236.126:43829)
0806 15:03:41.126519 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:41.126521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:41.126543 (+ 22us) server_negotiation.cc:407] Connection header received
0806 15:03:41.126598 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:41.126599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:41.126608 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:41.126663 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:41.126921 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.127626 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.128519 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.128786 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.128855 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:41.131144 (+ 2289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.65:49073: BlockingRecv error: recv got EOF from 127.0.236.65:49073 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 15:03:41.131403 13952 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 15:03:41.200419 13956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:41.200603 13956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:41.200634 13956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:41.201756 13956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:41.201794 13956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:41.201828 13956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:03:41.203220 13956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:34977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/oryx_127.0.236.66.keytab
--principal=oryx/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:40017
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=45977
--tserver_master_addrs=127.0.236.126:43829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:41.203444 13956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:41.203637 13956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:41.208189 13966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:41.208182 13967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:41.208164 13969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:41.208487 13956 server_base.cc:1034] running on GCE node
I20260806 15:03:41.208688 13956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:41.208910 13956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:41.210057 13956 hybrid_clock.cc:648] HybridClock initialized: now 1786028621210040 us; error 26 us; skew 500 ppm
Aug 06 15:03:41 dist-test-slave-tc2s krb5kdc[12884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028621, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:41.212399 13956 init.cc:377] Logged in from keytab as oryx/127.0.236.66@KRBTEST.COM (short username oryx)
I20260806 15:03:41.212774 13956 webserver.cc:460] Webserver started at http://127.0.236.66:45977/ using document root <none> and password file <none>
I20260806 15:03:41.212967 13956 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:41.213028 13956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:41.214293 13956 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:03:41.214797 13982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:41.214948 13956 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:41.215029 13956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "92a6bd7720ac4495b944e844f9ba5966"
format_stamp: "Formatted at 2026-08-06 15:03:40 on dist-test-slave-tc2s"
server_key: "8ad475949645517523eb9eb5d5e8b06f"
server_key_iv: "2d5e9b688683fbc56ddb4437759b87a3"
server_key_version: "encryptionkey@0"
I20260806 15:03:41.215301 13956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:41.226480 13956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:41.228402 13956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:41.228528 13956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:41.228716 13956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:41.228979 13956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:41.229012 13956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:41.229039 13956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:41.229063 13956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:41.234592 13956 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:40017
I20260806 15:03:41.234627 14140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:40017 every 8 connection(s)
I20260806 15:03:41.234911 13956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
Aug 06 15:03:41 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:41 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 15:03:41.241165 13955 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:41.235535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:55253 (local address 127.0.236.126:43829)
0806 15:03:41.235581 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:41.235583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:41.235784 (+ 201us) server_negotiation.cc:407] Connection header received
0806 15:03:41.236186 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:41.236187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:41.236200 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:41.236247 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:41.237067 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.237541 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.238469 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.238647 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.238689 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:41.241111 (+ 2422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.66:55253: BlockingRecv error: recv got EOF from 127.0.236.66:55253 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 15:03:41.241194 14143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:41.235538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.66:55253)
0806 15:03:41.235683 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:41.235714 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:41.236119 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:41.236293 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:41.236297 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:41.236821 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:41.236996 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:41.237000 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.237599 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.237600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:41.238391 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:41.238393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.238697 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.238697 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:41.238788 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:41.239132 (+ 344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:41.239139 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:41.241017 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":77,"threads_started":1}
W20260806 15:03:41.241372 14141 heartbeater.cc:644] Failed to heartbeat to 127.0.236.126:43829 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.236.126:43829: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
I20260806 15:03:41.244098 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 13956
Aug 06 15:03:41 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:41.244386 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:38069
--local_ip_for_outbound_sockets=127.0.236.67
--tserver_master_addrs=127.0.236.126:43829
--webserver_port=42287
--webserver_interface=127.0.236.67
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:34977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.236.67
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/oryx_127.0.236.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 15:03:41 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:41.245206 14143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:41.241585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.66:46951)
0806 15:03:41.241671 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:41.241679 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:41.241719 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:41.241818 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:41.241818 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:41.241952 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:41.241993 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:41.241994 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.242530 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.242531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:41.242991 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:41.242992 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.243240 (+ 248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.243241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:41.243281 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:41.243564 (+ 283us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:41.243567 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:41.245150 (+ 1583us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":73}
W20260806 15:03:41.245273 13955 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:41.241636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:46951 (local address 127.0.236.126:43829)
0806 15:03:41.241675 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:41.241677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:41.241709 (+ 32us) server_negotiation.cc:407] Connection header received
0806 15:03:41.241765 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:41.241766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:41.241776 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:41.241810 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:41.242048 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.242478 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.243038 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.243199 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.243239 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:41.245221 (+ 1982us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.66:46951: BlockingRecv error: recv got EOF from 127.0.236.66:46951 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 06 15:03:41 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:41 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:41.249221 14143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:41.245432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.66:38979)
0806 15:03:41.245470 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:41.245478 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:41.245508 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:41.245610 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:41.245611 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:41.245734 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:41.245764 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:41.245765 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.246532 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.246533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:41.246989 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:41.246991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.247350 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.247351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:41.247398 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:41.247712 (+ 314us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:41.247717 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:41.249167 (+ 1450us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 15:03:41.249300 13955 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:41.245477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:38979 (local address 127.0.236.126:43829)
0806 15:03:41.245519 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:41.245521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:41.245528 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:03:41.245545 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:41.245545 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:41.245552 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:41.245595 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:41.245815 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.246464 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.247058 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.247282 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.247339 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:41.249251 (+ 1912us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.66:38979: BlockingRecv error: recv got EOF from 127.0.236.66:38979 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260806 15:03:41.249346 14141 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 15:03:41.324761 14145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:41.324940 14145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:41.324962 14145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:41.326092 14145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:41.326128 14145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:41.326155 14145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:03:41.327533 14145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:34977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/krb5kdc/oryx_127.0.236.67.keytab
--principal=oryx/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:38069
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=42287
--tserver_master_addrs=127.0.236.126:43829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:41.327730 14145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:41.327919 14145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:41.332458 14154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:41.332473 14152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:41.332500 14157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:41.332583 14145 server_base.cc:1034] running on GCE node
I20260806 15:03:41.332852 14145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:41.333071 14145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:41.334208 14145 hybrid_clock.cc:648] HybridClock initialized: now 1786028621334196 us; error 28 us; skew 500 ppm
Aug 06 15:03:41 dist-test-slave-tc2s krb5kdc[12884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028621, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:41.336426 14145 init.cc:377] Logged in from keytab as oryx/127.0.236.67@KRBTEST.COM (short username oryx)
I20260806 15:03:41.336800 14145 webserver.cc:460] Webserver started at http://127.0.236.67:42287/ using document root <none> and password file <none>
I20260806 15:03:41.337006 14145 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:41.337069 14145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:41.338287 14145 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:03:41.338805 14170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:41.338944 14145 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:41.339013 14145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "642a40d30ca3499ebb247283efc01d91"
format_stamp: "Formatted at 2026-08-06 15:03:40 on dist-test-slave-tc2s"
server_key: "9aa47bac671fb3af54808d19f6934f59"
server_key_iv: "988ae1d8f0391aca46163930b4bb0a37"
server_key_version: "encryptionkey@0"
I20260806 15:03:41.339293 14145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:41.358345 14145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:41.359890 14145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:41.360015 14145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:41.360208 14145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:41.360493 14145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:41.360527 14145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:41.360555 14145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:41.360572 14145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:41.365823 14145 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:38069
I20260806 15:03:41.365857 14326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:38069 every 8 connection(s)
I20260806 15:03:41.366163 14145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
I20260806 15:03:41.370105 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 14145
Aug 06 15:03:41 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:41 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 15:03:41.373013 13955 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:41.366763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:52921 (local address 127.0.236.126:43829)
0806 15:03:41.366820 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:41.366823 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:41.367026 (+ 203us) server_negotiation.cc:407] Connection header received
0806 15:03:41.367448 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:41.367449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:41.367458 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:41.367506 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:41.368307 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.368779 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.369766 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.369994 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.370062 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:41.372967 (+ 2905us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.67:52921: BlockingRecv error: recv got EOF from 127.0.236.67:52921 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 15:03:41.373015 14329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:41.366753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.67:52921)
0806 15:03:41.366900 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:41.366946 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:41.367368 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:41.367554 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:41.367557 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:41.368071 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:41.368244 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:41.368248 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.368841 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.368842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:41.369668 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:41.369671 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.370063 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.370064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:41.370169 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:41.370640 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:41.370650 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:41.372878 (+ 2228us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":68,"threads_started":1}
W20260806 15:03:41.373188 14327 heartbeater.cc:644] Failed to heartbeat to 127.0.236.126:43829 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.236.126:43829: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Aug 06 15:03:41 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:41 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:41.377272 14329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:41.373368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.67:56933)
0806 15:03:41.373470 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:41.373481 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:41.373523 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:41.373624 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:41.373625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:41.373753 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:41.373794 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:41.373796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.374407 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.374408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:41.374965 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:41.374967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.375308 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.375309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:41.375353 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:41.375690 (+ 337us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:41.375694 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:41.377213 (+ 1519us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":85}
W20260806 15:03:41.377303 13955 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:41.373417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:56933 (local address 127.0.236.126:43829)
0806 15:03:41.373457 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:41.373459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:41.373511 (+ 52us) server_negotiation.cc:407] Connection header received
0806 15:03:41.373571 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:41.373571 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:41.373579 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:41.373618 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:41.373865 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.374333 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.375030 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.375255 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.375304 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:41.377265 (+ 1961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.67:56933: BlockingRecv error: recv got EOF from 127.0.236.67:56933 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:03:41.378181 14332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:41.370598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55250 (local address 127.0.236.126:43829)
0806 15:03:41.370681 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:41.370683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:41.370695 (+ 12us) server_negotiation.cc:407] Connection header received
0806 15:03:41.370773 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:41.370774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:41.370781 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:41.370827 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:41.371507 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.372133 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.373105 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.373362 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.373421 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:41.374631 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:41.374635 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:41.374635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:41.374645 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:41.376602 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:41.376911 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:41.376915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:41.376915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:41.377036 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:41.377400 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:41.377401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:41.377401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:41.377707 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:41.377833 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:41.377992 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 15:03:41.378097 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65,"thread_start_us":42,"threads_started":1}
Aug 06 15:03:41 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:41 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:41.381253 14329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:41.377568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.67:51403)
0806 15:03:41.377606 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:41.377614 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:41.377654 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:41.377758 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:41.377759 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:41.377870 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:41.377916 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:41.377917 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.378551 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.378552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:41.379101 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:41.379103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.379411 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.379412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:41.379473 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:41.379814 (+ 341us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:41.379819 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:41.381199 (+ 1380us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 15:03:41.381319 13955 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:41.377624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:51403 (local address 127.0.236.126:43829)
0806 15:03:41.377672 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:41.377673 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:41.377681 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:41.377699 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:41.377699 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:41.377707 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:41.377747 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:41.377968 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.378479 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.379165 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:41.379357 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:41.379406 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:41.381275 (+ 1869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.67:51403: BlockingRecv error: recv got EOF from 127.0.236.67:51403 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260806 15:03:41.381392 14327 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 15:03:42 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:42 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:42.135802 14335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:42.131725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.65:57737)
0806 15:03:42.131852 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:42.131864 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:42.131919 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:42.132059 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:42.132060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:42.132262 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:42.132324 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:42.132327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:42.132959 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:42.132960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:42.133489 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:42.133491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:42.133762 (+ 271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:42.133763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:42.133797 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:42.134185 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:42.134189 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:42.135738 (+ 1549us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":63,"threads_started":1}
W20260806 15:03:42.135886 14338 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:42.131804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:57737 (local address 127.0.236.126:43829)
0806 15:03:42.131922 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:42.131924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:42.131949 (+ 25us) server_negotiation.cc:407] Connection header received
0806 15:03:42.131975 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:42.131976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:42.131987 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:42.132046 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:42.132392 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:42.132884 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:42.133540 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:42.133719 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:42.133760 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:42.135824 (+ 2064us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.65:57737: BlockingRecv error: recv got EOF from 127.0.236.65:57737 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":55,"threads_started":1}
Aug 06 15:03:42 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:42 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:42.253684 14339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:42.249651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.66:41995)
0806 15:03:42.249753 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:42.249764 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:42.249818 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:42.249957 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:42.249958 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:42.250105 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:42.250152 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:42.250153 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:42.250729 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:42.250729 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:42.251264 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:42.251265 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:42.251535 (+ 270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:42.251536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:42.251572 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:42.251929 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:42.251933 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:42.253604 (+ 1671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":51,"threads_started":1}
W20260806 15:03:42.253753 14338 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:42.249701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:41995 (local address 127.0.236.126:43829)
0806 15:03:42.249767 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:42.249769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:42.249812 (+ 43us) server_negotiation.cc:407] Connection header received
0806 15:03:42.249854 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:42.249855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:42.249863 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:42.249952 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:42.250201 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:42.250678 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:42.251314 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:42.251491 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:42.251532 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:42.253700 (+ 2168us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.66:41995: BlockingRecv error: recv got EOF from 127.0.236.66:41995 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 06 15:03:42 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:42 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:42.385850 14340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:42.381715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.67:37061)
0806 15:03:42.381843 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:42.381852 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:42.381936 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:42.382048 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:42.382049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:42.382197 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:42.382244 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:42.382246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:42.382818 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:42.382819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:42.383342 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:42.383343 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:42.383614 (+ 271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:42.383614 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:42.383651 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:42.384029 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:42.384033 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:42.385789 (+ 1756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":69,"threads_started":1}
W20260806 15:03:42.385972 14338 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:42.381776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:37061 (local address 127.0.236.126:43829)
0806 15:03:42.381830 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:42.381832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:42.381905 (+ 73us) server_negotiation.cc:407] Connection header received
0806 15:03:42.381983 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:42.381983 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:42.381994 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:42.382038 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:42.382291 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:42.382769 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:42.383392 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:42.383566 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:42.383610 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:42.385879 (+ 2269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.67:37061: BlockingRecv error: recv got EOF from 127.0.236.67:37061 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 06 15:03:43 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:43 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:43.140195 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:43.136207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.65:45557)
0806 15:03:43.136323 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:43.136335 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:43.136393 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:43.136502 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:43.136502 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:43.136657 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:43.136707 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:43.136708 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:43.137306 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:43.137307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:43.137835 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:43.137837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:43.138131 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:43.138132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:43.138168 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:43.138566 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:43.138569 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:43.140138 (+ 1569us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":57,"threads_started":1}
W20260806 15:03:43.140295 14342 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:43.136268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:45557 (local address 127.0.236.126:43829)
0806 15:03:43.136385 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:43.136387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:43.136394 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:03:43.136437 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:43.136438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:43.136447 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:43.136497 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:43.136753 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:43.137248 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:43.137886 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:43.138085 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:43.138128 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:43.140223 (+ 2095us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.65:45557: BlockingRecv error: recv got EOF from 127.0.236.65:45557 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":72,"threads_started":1}
Aug 06 15:03:43 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:43 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:43.258211 14343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:43.254143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.66:55889)
0806 15:03:43.254280 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:43.254292 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:43.254362 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:43.254476 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:43.254476 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:43.254632 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:43.254682 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:43.254684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:43.255260 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:43.255260 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:43.255788 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:43.255790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:43.256060 (+ 270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:43.256060 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:43.256096 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:43.256486 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:43.256490 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:43.258156 (+ 1666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":53,"threads_started":1}
W20260806 15:03:43.258298 14342 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:43.254237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:55889 (local address 127.0.236.126:43829)
0806 15:03:43.254282 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:43.254284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:43.254345 (+ 61us) server_negotiation.cc:407] Connection header received
0806 15:03:43.254414 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:43.254414 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:43.254425 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:43.254461 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:43.254731 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:43.255210 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:43.255840 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:43.256014 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:43.256056 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:43.258238 (+ 2182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.66:55889: BlockingRecv error: recv got EOF from 127.0.236.66:55889 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 06 15:03:43 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:43 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:43.390519 14344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:43.386383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.67:35773)
0806 15:03:43.386495 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:43.386506 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:43.386556 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:43.386666 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:43.386666 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:43.386818 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:43.386870 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:43.386872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:43.387464 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:43.387464 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:43.387985 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:43.387986 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:43.388248 (+ 262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:43.388248 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:43.388283 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:43.388671 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:43.388675 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:43.390462 (+ 1787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":55,"threads_started":1}
W20260806 15:03:43.390597 14342 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:43.386489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:35773 (local address 127.0.236.126:43829)
0806 15:03:43.386538 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:43.386540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:43.386547 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:03:43.386607 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:43.386608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:43.386618 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:43.386655 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:43.386919 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:43.387401 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:43.388037 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:43.388204 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:43.388246 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:43.390539 (+ 2293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.67:35773: BlockingRecv error: recv got EOF from 127.0.236.67:35773 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 06 15:03:44 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:44 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:44.144773 14348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:44.140588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.65:34909)
0806 15:03:44.140711 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:44.140720 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:44.140764 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:44.140882 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:44.140882 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:44.141048 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:44.141098 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:44.141100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:44.141717 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:44.141717 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:44.142300 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:44.142302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:44.142610 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:44.142611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:44.142661 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:44.143098 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:44.143110 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:44.144714 (+ 1604us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":59,"threads_started":1}
W20260806 15:03:44.144873 14350 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:44.140643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:34909 (local address 127.0.236.126:43829)
0806 15:03:44.140752 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:44.140754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:44.140762 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:44.140805 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:44.140805 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:44.140816 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:44.140872 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:44.141145 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:44.141656 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:44.142360 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:44.142561 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:44.142608 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:44.144806 (+ 2198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.65:34909: BlockingRecv error: recv got EOF from 127.0.236.65:34909 (error 108)
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":57,"threads_started":1}
Aug 06 15:03:44 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:44 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:44.263185 14351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:44.258615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.66:49731)
0806 15:03:44.258733 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:44.258744 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:44.258795 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:44.258903 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:44.258904 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:44.259055 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:44.259104 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:44.259106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:44.259690 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:44.259690 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:44.260708 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:44.260712 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:44.261035 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:44.261035 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:44.261076 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:44.261469 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:44.261473 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:44.263128 (+ 1655us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":57,"threads_started":1}
W20260806 15:03:44.263281 14350 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:44.258669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:49731 (local address 127.0.236.126:43829)
0806 15:03:44.258721 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:44.258723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:44.258785 (+ 62us) server_negotiation.cc:407] Connection header received
0806 15:03:44.258834 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:44.258834 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:44.258845 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:44.258891 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:44.259153 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:44.259638 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:44.260787 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:44.260978 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:44.261025 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:44.263223 (+ 2198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.66:49731: BlockingRecv error: recv got EOF from 127.0.236.66:49731 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 06 15:03:44 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:44 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:44.395349 14352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:44.390904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.67:45397)
0806 15:03:44.391035 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:44.391045 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:44.391105 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:44.391222 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:44.391223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:44.391386 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:44.391435 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:44.391437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:44.392054 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:44.392055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:44.392804 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:44.392807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:44.393126 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:44.393126 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:44.393178 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:44.393702 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:44.393705 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:44.395275 (+ 1570us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":68,"threads_started":1}
W20260806 15:03:44.395431 14350 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:44.390958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:45397 (local address 127.0.236.126:43829)
0806 15:03:44.391006 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:44.391009 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:44.391096 (+ 87us) server_negotiation.cc:407] Connection header received
0806 15:03:44.391152 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:44.391153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:44.391165 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:44.391211 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:44.391500 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:44.391986 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:44.392875 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:44.393068 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:44.393121 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:44.395368 (+ 2247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.67:45397: BlockingRecv error: recv got EOF from 127.0.236.67:45397 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 06 15:03:45 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:45 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:45.149305 14354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:45.145167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.65:42441)
0806 15:03:45.145283 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:45.145292 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:45.145337 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:45.145456 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:45.145456 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:45.145613 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:45.145662 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:45.145664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:45.146320 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:45.146321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:45.146847 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:45.146848 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:45.147161 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:45.147162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:45.147197 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:45.147601 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:45.147605 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:45.149246 (+ 1641us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":64,"threads_started":1}
W20260806 15:03:45.149406 14356 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:45.145215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:42441 (local address 127.0.236.126:43829)
0806 15:03:45.145336 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:45.145338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:45.145367 (+ 29us) server_negotiation.cc:407] Connection header received
0806 15:03:45.145387 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:45.145387 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:45.145397 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:45.145451 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:45.145714 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:45.146257 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:45.146905 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:45.147115 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:45.147155 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:45.149333 (+ 2178us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.65:42441: BlockingRecv error: recv got EOF from 127.0.236.65:42441 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":60,"threads_started":1}
Aug 06 15:03:45 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:45 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:45.270539 14359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:45.263594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.66:49485)
0806 15:03:45.263732 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:45.263744 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:45.263809 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:45.263924 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:45.263925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:45.264104 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:45.264150 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:45.264152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:45.264791 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:45.264793 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:45.265320 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:45.265322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:45.265660 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:45.265661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:45.265719 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:45.268819 (+ 3100us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:45.268824 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:45.270480 (+ 1656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":69,"threads_started":1}
W20260806 15:03:45.270627 14356 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:45.263658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:49485 (local address 127.0.236.126:43829)
0806 15:03:45.263717 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:45.263719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:45.263791 (+ 72us) server_negotiation.cc:407] Connection header received
0806 15:03:45.263866 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:45.263867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:45.263877 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:45.263919 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:45.264224 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:45.264719 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:45.265399 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:45.265600 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:45.265654 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:45.270569 (+ 4915us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.66:49485: BlockingRecv error: recv got EOF from 127.0.236.66:49485 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 06 15:03:45 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:45 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:45.400168 14360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:45.395755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.67:36367)
0806 15:03:45.395909 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:45.395919 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:45.395977 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:45.396095 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:45.396096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:45.396261 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:45.396310 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:45.396312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:45.396925 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:45.396926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:45.397500 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:45.397503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:45.397854 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:45.397856 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:45.397954 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:45.398381 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:45.398385 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:45.400091 (+ 1706us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":81,"threads_started":1}
W20260806 15:03:45.400241 14356 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:45.395818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:36367 (local address 127.0.236.126:43829)
0806 15:03:45.395885 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:45.395887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:45.395969 (+ 82us) server_negotiation.cc:407] Connection header received
0806 15:03:45.396024 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:45.396024 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:45.396037 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:45.396085 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:45.396376 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:45.396855 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:45.397576 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:45.397793 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:45.397846 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:45.400187 (+ 2341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.67:36367: BlockingRecv error: recv got EOF from 127.0.236.67:36367 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 06 15:03:46 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:46 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:46.153673 14361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:46.149704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.65:42419)
0806 15:03:46.149828 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:46.149838 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:46.149903 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:46.150037 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:46.150038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:46.150197 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:46.150248 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:46.150249 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:46.150835 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:46.150836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:46.151353 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:46.151355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:46.151622 (+ 267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:46.151623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:46.151659 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:46.152047 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:46.152051 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:46.153615 (+ 1564us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":65,"threads_started":1}
W20260806 15:03:46.153771 14362 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:46.149762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:42419 (local address 127.0.236.126:43829)
0806 15:03:46.149860 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:46.149862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:46.149874 (+ 12us) server_negotiation.cc:407] Connection header received
0806 15:03:46.149960 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:46.149961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:46.149971 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:46.150020 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:46.150296 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:46.150785 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:46.151405 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:46.151579 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:46.151619 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:46.153707 (+ 2088us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.65:42419: BlockingRecv error: recv got EOF from 127.0.236.65:42419 (error 108)
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":49,"threads_started":1}
Aug 06 15:03:46 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:46 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:46.275105 14363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:46.270951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.66:41597)
0806 15:03:46.271084 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:46.271095 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:46.271163 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:46.271314 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:46.271315 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:46.271513 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:46.271575 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:46.271577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:46.272194 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:46.272194 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:46.272709 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:46.272710 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:46.272991 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:46.272991 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:46.273027 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:46.273438 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:46.273443 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:46.275048 (+ 1605us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":70,"threads_started":1}
W20260806 15:03:46.275193 14362 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:46.271043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:41597 (local address 127.0.236.126:43829)
0806 15:03:46.271099 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:46.271101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:46.271158 (+ 57us) server_negotiation.cc:407] Connection header received
0806 15:03:46.271229 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:46.271229 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:46.271240 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:46.271281 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:46.271640 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:46.272135 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:46.272764 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:46.272944 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:46.272988 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:46.275132 (+ 2144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.66:41597: BlockingRecv error: recv got EOF from 127.0.236.66:41597 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 06 15:03:46 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:46 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:46.404825 14364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:46.400569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.67:38991)
0806 15:03:46.400712 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:46.400724 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:46.400782 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:46.400912 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:46.400913 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:46.401085 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:46.401131 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:46.401133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:46.401756 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:46.401758 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:46.402332 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:46.402335 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:46.402663 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:46.402664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:46.402717 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:46.403129 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:46.403133 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:46.404768 (+ 1635us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
W20260806 15:03:46.404922 14362 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:46.400636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:38991 (local address 127.0.236.126:43829)
0806 15:03:46.400697 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:46.400699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:46.400774 (+ 75us) server_negotiation.cc:407] Connection header received
0806 15:03:46.400837 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:46.400838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:46.400851 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:46.400905 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:46.401200 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:46.401686 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:46.402409 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:46.402604 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:46.402659 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:46.404855 (+ 2196us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.67:38991: BlockingRecv error: recv got EOF from 127.0.236.67:38991 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 06 15:03:47 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:47 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:47.158313 14365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:47.154107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.65:45025)
0806 15:03:47.154273 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:47.154287 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:47.154349 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:47.154483 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:47.154483 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:47.154650 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:47.154696 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:47.154698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:47.155293 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:47.155294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:47.155828 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:47.155830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:47.156103 (+ 273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:47.156104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:47.156140 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:47.156549 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:47.156552 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:47.158255 (+ 1703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":71,"threads_started":1}
W20260806 15:03:47.158411 14366 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:47.154203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:45025 (local address 127.0.236.126:43829)
0806 15:03:47.154335 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:47.154337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:47.154345 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:47.154404 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:47.154404 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:47.154414 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:47.154467 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:47.154745 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:47.155241 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:47.155881 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:47.156057 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:47.156097 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:47.158345 (+ 2248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.65:45025: BlockingRecv error: recv got EOF from 127.0.236.65:45025 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":72,"threads_started":1}
Aug 06 15:03:47 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:47 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:47.279699 14367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:47.275502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.66:59503)
0806 15:03:47.275631 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:47.275642 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:47.275716 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:47.275849 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:47.275850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:47.276058 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:47.276118 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:47.276120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:47.276730 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:47.276731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:47.277244 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:47.277245 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:47.277532 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:47.277533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:47.277569 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:47.278009 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:47.278013 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:47.279643 (+ 1630us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
W20260806 15:03:47.279809 14366 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:47.275566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:59503 (local address 127.0.236.126:43829)
0806 15:03:47.275617 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:47.275618 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:47.275709 (+ 91us) server_negotiation.cc:407] Connection header received
0806 15:03:47.275767 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:47.275768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:47.275780 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:47.275838 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:47.276182 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:47.276673 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:47.277296 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:47.277484 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:47.277529 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:47.279741 (+ 2212us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.66:59503: BlockingRecv error: recv got EOF from 127.0.236.66:59503 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 06 15:03:47 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:47 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:47.410555 14370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:47.405205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.67:39911)
0806 15:03:47.405327 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:47.405340 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:47.405412 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:47.405566 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:47.405567 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:47.405772 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:47.405835 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:47.405838 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:47.406758 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:47.406759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:47.407590 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:47.407593 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:47.408054 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:47.408055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:47.408136 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:47.408599 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:47.408604 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:47.410497 (+ 1893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":64,"threads_started":1}
W20260806 15:03:47.410730 14366 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:47.405288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:39911 (local address 127.0.236.126:43829)
0806 15:03:47.405333 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:47.405335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:47.405421 (+ 86us) server_negotiation.cc:407] Connection header received
0806 15:03:47.405476 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:47.405477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:47.405490 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:47.405548 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:47.405916 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:47.406664 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:47.407689 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:47.407976 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:47.408052 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:47.410664 (+ 2612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.67:39911: BlockingRecv error: recv got EOF from 127.0.236.67:39911 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 06 15:03:48 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:48 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:48.162879 14371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:48.158727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.65:35511)
0806 15:03:48.158849 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:48.158859 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:48.158907 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:48.159039 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:48.159039 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:48.159191 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:48.159241 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:48.159243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:48.159837 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:48.159838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:48.160370 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:48.160371 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:48.160647 (+ 276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:48.160647 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:48.160684 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:48.161063 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:48.161067 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:48.162804 (+ 1737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":71,"threads_started":1}
W20260806 15:03:48.162969 14372 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:48.158800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:35511 (local address 127.0.236.126:43829)
0806 15:03:48.158937 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:48.158938 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:48.158946 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:48.158965 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:48.158966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:48.158976 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:48.159028 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:48.159290 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:48.159788 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:48.160427 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:48.160603 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:48.160644 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:48.162907 (+ 2263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.65:35511: BlockingRecv error: recv got EOF from 127.0.236.65:35511 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
Aug 06 15:03:48 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:48 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:48.284066 14374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:48.280077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.66:34707)
0806 15:03:48.280194 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:48.280205 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:48.280259 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:48.280380 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:48.280381 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:48.280534 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:48.280583 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:48.280584 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:48.281173 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:48.281173 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:48.281681 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:48.281682 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:48.281961 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:48.281962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:48.281999 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:48.282376 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:48.282380 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:48.284009 (+ 1629us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":64,"threads_started":1}
W20260806 15:03:48.284155 14372 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:48.280133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:34707 (local address 127.0.236.126:43829)
0806 15:03:48.280189 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:48.280191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:48.280254 (+ 63us) server_negotiation.cc:407] Connection header received
0806 15:03:48.280301 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:48.280301 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:48.280311 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:48.280374 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:48.280630 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:48.281122 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:48.281730 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:48.281921 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:48.281962 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:48.284094 (+ 2132us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.66:34707: BlockingRecv error: recv got EOF from 127.0.236.66:34707 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 06 15:03:48 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:48 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:48.415335 14376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:48.410990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.67:46913)
0806 15:03:48.411117 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:48.411129 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:48.411201 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:48.411360 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:48.411362 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:48.411562 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:48.411615 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:48.411617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:48.412279 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:48.412280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:48.412816 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:48.412818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:48.413140 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:48.413141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:48.413194 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:48.413596 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:48.413601 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:48.415279 (+ 1678us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
W20260806 15:03:48.415427 14372 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:48.411052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:46913 (local address 127.0.236.126:43829)
0806 15:03:48.411095 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:48.411097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:48.411197 (+ 100us) server_negotiation.cc:407] Connection header received
0806 15:03:48.411250 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:48.411251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:48.411266 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:48.411318 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:48.411686 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:48.412192 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:48.412893 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:48.413082 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:48.413135 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:48.415367 (+ 2232us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.67:46913: BlockingRecv error: recv got EOF from 127.0.236.67:46913 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 06 15:03:49 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:49 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:49.167346 14378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:49.163310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.65:42025)
0806 15:03:49.163464 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:49.163476 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:49.163532 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:49.163646 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:49.163647 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:49.163799 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:49.163848 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:49.163849 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:49.164439 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:49.164440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:49.164962 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:49.164963 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:49.165238 (+ 275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:49.165239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:49.165273 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:49.165676 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:49.165680 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:49.167279 (+ 1599us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":88,"threads_started":1}
W20260806 15:03:49.167451 14382 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:49.163417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:42025 (local address 127.0.236.126:43829)
0806 15:03:49.163528 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:49.163530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:49.163537 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:03:49.163572 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:49.163573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:49.163584 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:49.163635 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:49.163897 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:49.164388 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:49.165014 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:49.165193 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:49.165234 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:49.167367 (+ 2133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.65:42025: BlockingRecv error: recv got EOF from 127.0.236.65:42025 (error 108)
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":63,"threads_started":1}
Aug 06 15:03:49 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:49 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:49.288654 14385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:49.284478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.66:43417)
0806 15:03:49.284622 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:49.284631 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:49.284683 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:49.284792 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:49.284793 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:49.284944 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:49.284996 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:49.284997 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:49.285756 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:49.285757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:49.286294 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:49.286297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:49.286600 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:49.286601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:49.286642 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:49.287030 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:49.287033 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:49.288584 (+ 1551us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":66,"threads_started":1}
W20260806 15:03:49.288756 14382 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:49.284533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:43417 (local address 127.0.236.126:43829)
0806 15:03:49.284592 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:49.284594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:49.284677 (+ 83us) server_negotiation.cc:407] Connection header received
0806 15:03:49.284724 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:49.284725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:49.284736 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:49.284782 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:49.285042 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:49.285704 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:49.286355 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:49.286545 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:49.286591 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:49.288692 (+ 2101us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.66:43417: BlockingRecv error: recv got EOF from 127.0.236.66:43417 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 06 15:03:49 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:49 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:49.419867 14389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:49.415773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.67:45143)
0806 15:03:49.415902 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:49.415912 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:49.415970 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:49.416112 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:49.416114 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:49.416272 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:49.416322 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:49.416324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:49.416911 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:49.416912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:49.417433 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:49.417435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:49.417697 (+ 262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:49.417698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:49.417743 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:49.418156 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:49.418161 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:49.419797 (+ 1636us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":63,"threads_started":1}
W20260806 15:03:49.419945 14382 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:49.415804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:45143 (local address 127.0.236.126:43829)
0806 15:03:49.415848 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:49.415850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:49.415953 (+ 103us) server_negotiation.cc:407] Connection header received
0806 15:03:49.416023 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:49.416024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:49.416045 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:49.416095 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:49.416378 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:49.416859 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:49.417484 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:49.417655 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:49.417695 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:49.419882 (+ 2187us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.67:45143: BlockingRecv error: recv got EOF from 127.0.236.67:45143 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
Aug 06 15:03:50 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:50 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:50.172058 14390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:50.167740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.65:39505)
0806 15:03:50.167874 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:50.167887 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:50.167948 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:50.168065 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:50.168066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:50.168223 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:50.168273 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:50.168274 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:50.168901 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:50.168902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:50.169443 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:50.169446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:50.169790 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:50.169790 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:50.169827 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:50.170269 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:50.170273 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:50.171997 (+ 1724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
W20260806 15:03:50.172163 14391 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:50.167792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:39505 (local address 127.0.236.126:43829)
0806 15:03:50.167926 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:50.167927 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:50.167935 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:50.167990 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:50.167991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:50.168001 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:50.168055 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:50.168320 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:50.168846 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:50.169519 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:50.169739 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:50.169785 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:50.172086 (+ 2301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.65:39505: BlockingRecv error: recv got EOF from 127.0.236.65:39505 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":70,"threads_started":1}
Aug 06 15:03:50 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:50 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:50.297837 14392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:50.289156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.66:33101)
0806 15:03:50.289284 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:50.289294 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:50.289345 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:50.289457 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:50.289458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:50.289610 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:50.289662 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:50.289663 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:50.290287 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:50.290288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:50.290973 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:50.290976 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:50.291277 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:50.291277 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:50.291317 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:50.296172 (+ 4855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:50.296177 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:50.297759 (+ 1582us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
W20260806 15:03:50.298041 14391 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:50.289228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:33101 (local address 127.0.236.126:43829)
0806 15:03:50.289272 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:50.289274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:50.289333 (+ 59us) server_negotiation.cc:407] Connection header received
0806 15:03:50.289383 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:50.289384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:50.289395 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:50.289447 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:50.289710 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:50.290218 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:50.291033 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:50.291225 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:50.291273 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:50.297982 (+ 6709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.66:33101: BlockingRecv error: recv got EOF from 127.0.236.66:33101 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 06 15:03:50 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:50 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:50.424306 14393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:50.420265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.67:34907)
0806 15:03:50.420420 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:50.420431 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:50.420495 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:50.420599 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:50.420600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:50.420753 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:50.420801 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:50.420803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:50.421368 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:50.421369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:50.421884 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:50.421885 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:50.422185 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:50.422185 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:50.422221 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:50.422608 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:50.422612 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:50.424248 (+ 1636us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":96,"threads_started":1}
W20260806 15:03:50.424387 14391 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:50.420331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:34907 (local address 127.0.236.126:43829)
0806 15:03:50.420377 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:50.420379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:50.420490 (+ 111us) server_negotiation.cc:407] Connection header received
0806 15:03:50.420543 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:50.420544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:50.420555 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:50.420588 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:50.420844 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:50.421318 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:50.421964 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:50.422139 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:50.422181 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:50.424331 (+ 2150us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.67:34907: BlockingRecv error: recv got EOF from 127.0.236.67:34907 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 06 15:03:51 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:51 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:51.176584 14394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:51.172467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.65:45903)
0806 15:03:51.172584 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:51.172596 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:51.172654 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:51.172762 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:51.172762 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:51.172919 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:51.172967 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:51.172968 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:51.173562 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:51.173563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:51.174114 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:51.174116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:51.174434 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:51.174435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:51.174471 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:51.174863 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:51.174867 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:51.176513 (+ 1646us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":55,"threads_started":1}
W20260806 15:03:51.176677 14395 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:51.172526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:45903 (local address 127.0.236.126:43829)
0806 15:03:51.172638 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:51.172640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:51.172648 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:51.172695 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:51.172696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:51.172706 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:51.172751 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:51.173012 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:51.173512 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:51.174179 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:51.174380 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:51.174423 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:51.176605 (+ 2182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.65:45903: BlockingRecv error: recv got EOF from 127.0.236.65:45903 (error 108)
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":67,"threads_started":1}
Aug 06 15:03:51 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:51 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:51.302583 14397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:51.298240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.66:44447)
0806 15:03:51.298372 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:51.298383 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:51.298442 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:51.298573 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:51.298574 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:51.298744 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:51.298791 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:51.298793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:51.299448 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:51.299449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:51.299977 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:51.299978 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:51.300304 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:51.300305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:51.300374 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:51.300806 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:51.300810 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:51.302497 (+ 1687us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
W20260806 15:03:51.302652 14395 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:51.298276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:44447 (local address 127.0.236.126:43829)
0806 15:03:51.298333 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:51.298335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:51.298433 (+ 98us) server_negotiation.cc:407] Connection header received
0806 15:03:51.298499 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:51.298500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:51.298512 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:51.298561 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:51.298855 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:51.299375 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:51.300051 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:51.300246 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:51.300299 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:51.302599 (+ 2300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.66:44447: BlockingRecv error: recv got EOF from 127.0.236.66:44447 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 06 15:03:51 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:51 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:51.428807 14398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:51.424694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.67:57245)
0806 15:03:51.424811 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:51.424821 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:51.424866 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:51.424983 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:51.424984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:51.425134 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:51.425184 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:51.425185 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:51.425755 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:51.425756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:51.426306 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:51.426308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:51.426589 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:51.426590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:51.426628 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:51.427012 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:51.427016 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:51.428732 (+ 1716us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":58,"threads_started":1}
W20260806 15:03:51.428902 14395 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:51.424799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:57245 (local address 127.0.236.126:43829)
0806 15:03:51.424846 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:51.424849 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:51.424857 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:51.424911 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:51.424912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:51.424922 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:51.424973 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:51.425229 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:51.425707 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:51.426361 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:51.426540 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:51.426587 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:51.428831 (+ 2244us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.67:57245: BlockingRecv error: recv got EOF from 127.0.236.67:57245 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 06 15:03:52 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:52 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:52.181207 14399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:52.176990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.65:43189)
0806 15:03:52.177127 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:52.177137 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:52.177203 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:52.177328 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:52.177329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:52.177493 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:52.177540 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:52.177542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:52.178164 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:52.178165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:52.178699 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:52.178701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:52.179026 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:52.179028 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:52.179092 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:52.179552 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:52.179556 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:52.181141 (+ 1585us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":70,"threads_started":1}
W20260806 15:03:52.181296 14400 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:52.177044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:43189 (local address 127.0.236.126:43829)
0806 15:03:52.177147 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:52.177149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:52.177196 (+ 47us) server_negotiation.cc:407] Connection header received
0806 15:03:52.177259 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:52.177260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:52.177270 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:52.177316 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:52.177587 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:52.178112 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:52.178756 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:52.178960 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:52.179011 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:52.181233 (+ 2222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.65:43189: BlockingRecv error: recv got EOF from 127.0.236.65:43189 (error 108)
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":46,"threads_started":1}
Aug 06 15:03:52 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:52 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:52.307541 14401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:52.303018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.66:35145)
0806 15:03:52.303161 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:52.303172 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:52.303234 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:52.303369 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:52.303370 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:52.303537 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:52.303584 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:52.303586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:52.304221 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:52.304222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:52.304778 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:52.304780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:52.305263 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:52.305264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:52.305327 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:52.305787 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:52.305792 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:52.307473 (+ 1681us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":63,"threads_started":1}
W20260806 15:03:52.307633 14400 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:52.303074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:35145 (local address 127.0.236.126:43829)
0806 15:03:52.303134 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:52.303136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:52.303228 (+ 92us) server_negotiation.cc:407] Connection header received
0806 15:03:52.303298 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:52.303299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:52.303311 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:52.303356 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:52.303656 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:52.304150 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:52.304884 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:52.305151 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:52.305222 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:52.307569 (+ 2347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.66:35145: BlockingRecv error: recv got EOF from 127.0.236.66:35145 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 06 15:03:52 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:52 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:52.433393 14402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:52.429238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.67:40389)
0806 15:03:52.429370 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:52.429380 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:52.429444 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:52.429554 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:52.429555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:52.429711 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:52.429759 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:52.429760 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:52.430362 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:52.430362 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:52.430905 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:52.430908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:52.431177 (+ 269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:52.431178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:52.431214 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:52.431609 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:52.431628 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:52.433324 (+ 1696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
W20260806 15:03:52.433467 14400 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:52.429296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:40389 (local address 127.0.236.126:43829)
0806 15:03:52.429350 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:52.429352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:52.429439 (+ 87us) server_negotiation.cc:407] Connection header received
0806 15:03:52.429487 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:52.429487 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:52.429498 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:52.429541 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:52.429803 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:52.430309 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:52.430963 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:52.431133 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:52.431173 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:52.433411 (+ 2238us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.67:40389: BlockingRecv error: recv got EOF from 127.0.236.67:40389 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 06 15:03:53 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:53 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:53.185786 14403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:53.181625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.65:41119)
0806 15:03:53.181751 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:53.181761 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:53.181807 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:53.181970 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:53.181971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:53.182134 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:53.182181 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:53.182183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:53.182779 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:53.182780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:53.183307 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:53.183308 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:53.183582 (+ 274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:53.183583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:53.183618 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:53.184051 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:53.184056 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:53.185725 (+ 1669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
W20260806 15:03:53.185927 14404 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:53.181694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:41119 (local address 127.0.236.126:43829)
0806 15:03:53.181792 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:53.181794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:53.181803 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:03:53.181848 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:53.181848 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:53.181858 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:53.181964 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:53.182234 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:53.182727 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:53.183358 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:53.183538 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:53.183579 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:53.185807 (+ 2228us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.65:41119: BlockingRecv error: recv got EOF from 127.0.236.65:41119 (error 108)
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":50,"threads_started":1}
Aug 06 15:03:53 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:53 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:53.312189 14405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:53.307949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.66:38701)
0806 15:03:53.308079 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:53.308091 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:53.308153 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:53.308282 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:53.308283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:53.308462 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:53.308506 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:53.308508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:53.309142 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:53.309143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:53.309662 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:53.309664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:53.310041 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:53.310042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:53.310102 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:53.310541 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:53.310547 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:53.312131 (+ 1584us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
W20260806 15:03:53.312276 14404 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:53.308005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:38701 (local address 127.0.236.126:43829)
0806 15:03:53.308064 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:53.308066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:53.308145 (+ 79us) server_negotiation.cc:407] Connection header received
0806 15:03:53.308216 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:53.308216 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:53.308228 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:53.308271 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:53.308577 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:53.309070 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:53.309736 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:53.309972 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:53.310024 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:53.312215 (+ 2191us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.66:38701: BlockingRecv error: recv got EOF from 127.0.236.66:38701 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 06 15:03:53 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:53 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:53.438088 14406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:53.433795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.67:46621)
0806 15:03:53.433951 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:53.433963 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:53.434025 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:53.434153 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:53.434154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:53.434322 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:53.434368 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:53.434370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:53.435007 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:53.435009 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:53.435524 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:53.435525 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:53.435841 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:53.435842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:53.435897 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:53.436326 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:53.436330 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:53.438012 (+ 1682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":68,"threads_started":1}
W20260806 15:03:53.438153 14404 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:53.433877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:46621 (local address 127.0.236.126:43829)
0806 15:03:53.433937 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:53.433940 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:53.434015 (+ 75us) server_negotiation.cc:407] Connection header received
0806 15:03:53.434074 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:53.434074 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:53.434087 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:53.434130 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:53.434448 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:53.434936 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:53.435599 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:53.435783 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:53.435836 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:53.438098 (+ 2262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.67:46621: BlockingRecv error: recv got EOF from 127.0.236.67:46621 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 06 15:03:54 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:54 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:54.190327 14408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:54.186184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.65:48329)
0806 15:03:54.186305 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:54.186316 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:54.186370 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:54.186503 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:54.186504 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:54.186665 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:54.186711 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:54.186712 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:54.187315 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:54.187316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:54.187842 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:54.187843 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:54.188122 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:54.188123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:54.188158 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:54.188592 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:54.188597 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:54.190258 (+ 1661us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":71,"threads_started":1}
W20260806 15:03:54.190434 14409 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:54.186240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:48329 (local address 127.0.236.126:43829)
0806 15:03:54.186395 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:54.186397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:54.186406 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:03:54.186426 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:54.186427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:54.186437 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:54.186493 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:54.186765 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:54.187264 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:54.187896 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:54.188077 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:54.188118 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:54.190358 (+ 2240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.65:48329: BlockingRecv error: recv got EOF from 127.0.236.65:48329 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":77,"threads_started":1}
Aug 06 15:03:54 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:54 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:54.316758 14410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:54.312579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.66:54823)
0806 15:03:54.312689 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:54.312700 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:54.312756 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:54.312877 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:54.312878 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:54.313046 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:54.313092 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:54.313094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:54.313737 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:54.313738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:54.314328 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:54.314330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:54.314673 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:54.314675 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:54.314733 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:54.315123 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:54.315127 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:54.316672 (+ 1545us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":53,"threads_started":1}
W20260806 15:03:54.316910 14409 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:54.312681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:54823 (local address 127.0.236.126:43829)
0806 15:03:54.312738 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:54.312740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:54.312749 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:03:54.312807 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:54.312808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:54.312820 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:54.312862 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:54.313156 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:54.313665 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:54.314402 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:54.314611 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:54.314667 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:54.316857 (+ 2190us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.66:54823: BlockingRecv error: recv got EOF from 127.0.236.66:54823 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 06 15:03:54 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:54 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:54.442746 14413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:54.438485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.67:35781)
0806 15:03:54.438627 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:54.438638 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:54.438696 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:54.438814 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:54.438815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:54.438993 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:54.439038 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:54.439040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:54.439669 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:54.439670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:54.440206 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:54.440208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:54.440543 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:54.440544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:54.440601 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:54.440996 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:54.441000 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:54.442665 (+ 1665us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
W20260806 15:03:54.442821 14409 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:54.438538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:35781 (local address 127.0.236.126:43829)
0806 15:03:54.438588 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:54.438590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:54.438686 (+ 96us) server_negotiation.cc:407] Connection header received
0806 15:03:54.438754 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:54.438755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:54.438767 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:54.438809 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:54.439108 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:54.439597 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:54.440280 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:54.440482 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:54.440536 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:54.442758 (+ 2222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.67:35781: BlockingRecv error: recv got EOF from 127.0.236.67:35781 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 06 15:03:55 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:55 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:55.194901 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:55.190715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.65:39949)
0806 15:03:55.190841 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:55.190852 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:55.190898 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:55.191033 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:55.191034 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:55.191193 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:55.191239 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:55.191240 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:55.191828 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:55.191829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:55.192354 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:55.192356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:55.192641 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:55.192641 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:55.192676 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:55.193085 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:55.193089 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:55.194822 (+ 1733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":57,"threads_started":1}
W20260806 15:03:55.194990 14415 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:55.190766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:39949 (local address 127.0.236.126:43829)
0806 15:03:55.190896 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:55.190897 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:55.190905 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:55.190956 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:55.190957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:55.190967 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:55.191023 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:55.191288 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:55.191779 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:55.192408 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:55.192596 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:55.192638 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:55.194919 (+ 2281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.65:39949: BlockingRecv error: recv got EOF from 127.0.236.65:39949 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
Aug 06 15:03:55 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:55 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:55.325421 14416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:55.317194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.66:52823)
0806 15:03:55.317343 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:55.317354 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:55.317417 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:55.317555 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:55.317556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:55.317730 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:55.317777 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:55.317778 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:55.318439 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:55.318440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:55.318964 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:55.318966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:55.319288 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:55.319289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:55.319347 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:55.323640 (+ 4293us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:55.323646 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:55.325346 (+ 1700us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":67,"threads_started":1}
W20260806 15:03:55.325511 14415 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:55.317249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:52823 (local address 127.0.236.126:43829)
0806 15:03:55.317307 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:55.317309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:55.317420 (+ 111us) server_negotiation.cc:407] Connection header received
0806 15:03:55.317480 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:55.317481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:55.317494 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:55.317541 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:55.317847 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:55.318368 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:55.319037 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:55.319229 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:55.319284 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:55.325456 (+ 6172us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.66:52823: BlockingRecv error: recv got EOF from 127.0.236.66:52823 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 06 15:03:55 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:55 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.67@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:55.447341 14417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:55.443121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.67:36323)
0806 15:03:55.443248 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:55.443260 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:55.443328 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:55.443466 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:55.443467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:55.443648 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:55.443698 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:55.443700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:55.444339 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:55.444340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:55.444865 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:55.444867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:55.445201 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:55.445202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:55.445255 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:55.445654 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:55.445658 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:55.447263 (+ 1605us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":65,"threads_started":1}
W20260806 15:03:55.447408 14415 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:55.443195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:36323 (local address 127.0.236.126:43829)
0806 15:03:55.443256 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:55.443259 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:55.443320 (+ 61us) server_negotiation.cc:407] Connection header received
0806 15:03:55.443386 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:55.443387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:55.443401 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:55.443451 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:55.443776 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:55.444268 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:55.444954 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:55.445141 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:55.445195 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:55.447354 (+ 2159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.67:36323: BlockingRecv error: recv got EOF from 127.0.236.67:36323 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 06 15:03:56 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:56 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.65@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:56.199536 14419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:56.195349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.65:57433)
0806 15:03:56.195495 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:56.195508 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:56.195565 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:56.195683 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:56.195684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:56.195842 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:56.195891 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:56.195893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:56.196478 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:56.196478 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:56.197019 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:56.197021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:56.197295 (+ 274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:56.197295 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:56.197331 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:56.197739 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:56.197743 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:56.199468 (+ 1725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
W20260806 15:03:56.199641 14421 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:56.195418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:57433 (local address 127.0.236.126:43829)
0806 15:03:56.195554 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:56.195556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:56.195564 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:56.195608 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:56.195609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:56.195619 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:56.195672 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:56.195940 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:56.196428 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:56.197069 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:56.197251 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:56.197292 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:56.199551 (+ 2259us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.65:57433: BlockingRecv error: recv got EOF from 127.0.236.65:57433 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
Aug 06 15:03:56 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:03:56 dist-test-slave-tc2s krb5kdc[12884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.236.66@KRBTEST.COM for oryx/127.0.236.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:03:56.330127 14422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:56.325878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43829 (local address 127.0.236.66:37143)
0806 15:03:56.326015 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:56.326025 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:56.326078 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:56.326197 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:56.326198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:56.326352 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:56.326419 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:56.326420 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:56.327030 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:56.327031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:56.327624 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:56.327627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:56.328065 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:56.328066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:56.328114 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:56.328514 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:56.328518 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:56.330058 (+ 1540us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.236.126:43829: Server oryx/127.0.236.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
W20260806 15:03:56.330240 14421 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:56.325926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:37143 (local address 127.0.236.126:43829)
0806 15:03:56.325979 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:56.325982 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:56.326073 (+ 91us) server_negotiation.cc:407] Connection header received
0806 15:03:56.326137 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:56.326138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:56.326148 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:56.326187 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:56.326467 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:56.326966 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:56.327687 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:56.327993 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:56.328046 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:56.330172 (+ 2126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.236.66:37143: BlockingRecv error: recv got EOF from 127.0.236.66:37143 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 15:03:56.380280 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 13739
I20260806 15:03:56.393151 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 13956
I20260806 15:03:56.401495 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 14145
I20260806 15:03:56.407217 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 13642
2026-08-06T15:03:56Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17455 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T15:03:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:03:56Z Disabled control of system clock
I20260806 15:03:56.437863 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:42613
--webserver_interface=127.0.236.126
--webserver_port=0
--builtin_ntp_servers=127.0.236.84:39355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:42613
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 15:03:56.517268 14427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:56.517459 14427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:56.517478 14427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:56.518580 14427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:03:56.518615 14427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:56.518626 14427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:56.518636 14427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:03:56.518652 14427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:03:56.519906 14427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:39355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:42613
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:42613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:56.520087 14427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:56.520304 14427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 15:03:56.524837 14427 server_base.cc:1034] running on GCE node
W20260806 15:03:56.524855 14439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:56.524806 14437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:56.524809 14434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:56.525231 14427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:56.525491 14427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:56.526618 14427 hybrid_clock.cc:648] HybridClock initialized: now 1786028636526609 us; error 28 us; skew 500 ppm
I20260806 15:03:56.527733 14427 webserver.cc:460] Webserver started at http://127.0.236.126:40419/ using document root <none> and password file <none>
I20260806 15:03:56.527956 14427 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:56.528000 14427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:56.528128 14427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:56.529109 14427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "92c44f114b0e4bdfb1613fb22d6f0796"
format_stamp: "Formatted at 2026-08-06 15:03:56 on dist-test-slave-tc2s"
server_key: "dca4f80295983101549dfff9734e6991"
server_key_iv: "09e2bb094c09519790529d4ef586316d"
server_key_version: "encryptionkey@0"
I20260806 15:03:56.529423 14427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "92c44f114b0e4bdfb1613fb22d6f0796"
format_stamp: "Formatted at 2026-08-06 15:03:56 on dist-test-slave-tc2s"
server_key: "dca4f80295983101549dfff9734e6991"
server_key_iv: "09e2bb094c09519790529d4ef586316d"
server_key_version: "encryptionkey@0"
I20260806 15:03:56.530632 14427 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:03:56.531306 14447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:56.531493 14427 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:03:56.531589 14427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "92c44f114b0e4bdfb1613fb22d6f0796"
format_stamp: "Formatted at 2026-08-06 15:03:56 on dist-test-slave-tc2s"
server_key: "dca4f80295983101549dfff9734e6991"
server_key_iv: "09e2bb094c09519790529d4ef586316d"
server_key_version: "encryptionkey@0"
I20260806 15:03:56.531664 14427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:56.560210 14427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:56.560560 14427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:56.560698 14427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:56.565132 14427 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:42613
I20260806 15:03:56.565202 14536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:42613 every 8 connection(s)
I20260806 15:03:56.565510 14427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:03:56.566308 14538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:56.569960 14538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92c44f114b0e4bdfb1613fb22d6f0796: Bootstrap starting.
I20260806 15:03:56.570722 14538 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92c44f114b0e4bdfb1613fb22d6f0796: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:56.571055 14538 log.cc:824] T 00000000000000000000000000000000 P 92c44f114b0e4bdfb1613fb22d6f0796: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:56.571846 14538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92c44f114b0e4bdfb1613fb22d6f0796: No bootstrap required, opened a new log
I20260806 15:03:56.571991 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 14427
I20260806 15:03:56.572081 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:03:56.572280 945 external_mini_cluster.cc:1428] Setting key f68ed228bfb21b2b7eb7d5d3596443bb
I20260806 15:03:56.574222 14538 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92c44f114b0e4bdfb1613fb22d6f0796 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92c44f114b0e4bdfb1613fb22d6f0796" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 42613 } }
I20260806 15:03:56.574414 14538 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92c44f114b0e4bdfb1613fb22d6f0796 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:56.574455 14538 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92c44f114b0e4bdfb1613fb22d6f0796 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92c44f114b0e4bdfb1613fb22d6f0796, State: Initialized, Role: FOLLOWER
I20260806 15:03:56.574606 14538 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92c44f114b0e4bdfb1613fb22d6f0796 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92c44f114b0e4bdfb1613fb22d6f0796" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 42613 } }
I20260806 15:03:56.574693 14538 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92c44f114b0e4bdfb1613fb22d6f0796 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:03:56.574728 14538 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92c44f114b0e4bdfb1613fb22d6f0796 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:03:56.574775 14538 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92c44f114b0e4bdfb1613fb22d6f0796 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:56.575451 14538 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92c44f114b0e4bdfb1613fb22d6f0796 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92c44f114b0e4bdfb1613fb22d6f0796" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 42613 } }
I20260806 15:03:56.575568 14538 leader_election.cc:302] T 00000000000000000000000000000000 P 92c44f114b0e4bdfb1613fb22d6f0796 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92c44f114b0e4bdfb1613fb22d6f0796; no voters:
I20260806 15:03:56.575729 14538 leader_election.cc:288] T 00000000000000000000000000000000 P 92c44f114b0e4bdfb1613fb22d6f0796 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:03:56.575815 14547 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92c44f114b0e4bdfb1613fb22d6f0796 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:56.575986 14538 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92c44f114b0e4bdfb1613fb22d6f0796 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:03:56.576006 14547 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92c44f114b0e4bdfb1613fb22d6f0796 [term 1 LEADER]: Becoming Leader. State: Replica: 92c44f114b0e4bdfb1613fb22d6f0796, State: Running, Role: LEADER
I20260806 15:03:56.577986 14547 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92c44f114b0e4bdfb1613fb22d6f0796 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92c44f114b0e4bdfb1613fb22d6f0796" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 42613 } }
I20260806 15:03:56.578583 14549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92c44f114b0e4bdfb1613fb22d6f0796 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92c44f114b0e4bdfb1613fb22d6f0796" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92c44f114b0e4bdfb1613fb22d6f0796" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 42613 } } }
I20260806 15:03:56.578670 14549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92c44f114b0e4bdfb1613fb22d6f0796 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:56.578841 14550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92c44f114b0e4bdfb1613fb22d6f0796 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92c44f114b0e4bdfb1613fb22d6f0796. Latest consensus state: current_term: 1 leader_uuid: "92c44f114b0e4bdfb1613fb22d6f0796" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92c44f114b0e4bdfb1613fb22d6f0796" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 42613 } } }
I20260806 15:03:56.578922 14550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92c44f114b0e4bdfb1613fb22d6f0796 [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:56.578881 14566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:03:56.579337 14544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:56.572796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36178 (local address 127.0.236.126:42613)
0806 15:03:56.572998 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:56.573006 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:56.573030 (+ 24us) server_negotiation.cc:407] Connection header received
0806 15:03:56.574173 (+ 1143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:56.574183 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:56.574274 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:56.574572 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:56.574689 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:56.576103 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:56.577156 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:56.577681 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:56.577793 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:56.578500 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:56.578505 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:56.578505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:03:56.578712 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:56.578762 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:03:56.578778 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:56.578849 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:56.579051 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 15:03:56.579139 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":80,"threads_started":1}
I20260806 15:03:56.579468 14566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:03:56.580999 14566 catalog_manager.cc:1347] Generated new cluster ID: 536dfea9300744df8b9c53d786784c57
I20260806 15:03:56.581058 14566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:03:56.585733 14566 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:03:56.586241 14566 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:03:56.593734 14566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92c44f114b0e4bdfb1613fb22d6f0796: Generated new TSK 0
I20260806 15:03:56.593883 14566 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:03:56.599505 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:0
--local_ip_for_outbound_sockets=127.0.236.65
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:42613
--builtin_ntp_servers=127.0.236.84:39355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 15:03:56.695879 14571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:56.696072 14571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:56.696099 14571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:56.697196 14571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:56.697233 14571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:56.697265 14571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:03:56.698626 14571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:39355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:42613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:56.698837 14571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:56.699102 14571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:56.703771 14582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:56.703758 14578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:56.703893 14580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:56.704123 14571 server_base.cc:1034] running on GCE node
I20260806 15:03:56.704311 14571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:56.704536 14571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:56.705679 14571 hybrid_clock.cc:648] HybridClock initialized: now 1786028636705666 us; error 31 us; skew 500 ppm
I20260806 15:03:56.706938 14571 webserver.cc:460] Webserver started at http://127.0.236.65:39537/ using document root <none> and password file <none>
I20260806 15:03:56.707171 14571 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:56.707219 14571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:56.707355 14571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:56.708365 14571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/data/instance:
uuid: "20564efbdaca418dab05790a436c8d02"
format_stamp: "Formatted at 2026-08-06 15:03:56 on dist-test-slave-tc2s"
server_key: "8622b6ab5d2fb8e25a785d627c3d6daf"
server_key_iv: "2d976875d660bed435eda5b2edf7407e"
server_key_version: "encryptionkey@0"
I20260806 15:03:56.708691 14571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/wal/instance:
uuid: "20564efbdaca418dab05790a436c8d02"
format_stamp: "Formatted at 2026-08-06 15:03:56 on dist-test-slave-tc2s"
server_key: "8622b6ab5d2fb8e25a785d627c3d6daf"
server_key_iv: "2d976875d660bed435eda5b2edf7407e"
server_key_version: "encryptionkey@0"
I20260806 15:03:56.710013 14571 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:03:56.710749 14589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:56.710912 14571 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:56.710994 14571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "20564efbdaca418dab05790a436c8d02"
format_stamp: "Formatted at 2026-08-06 15:03:56 on dist-test-slave-tc2s"
server_key: "8622b6ab5d2fb8e25a785d627c3d6daf"
server_key_iv: "2d976875d660bed435eda5b2edf7407e"
server_key_version: "encryptionkey@0"
I20260806 15:03:56.711061 14571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:56.732791 14571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:56.733132 14571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:56.733256 14571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:56.733479 14571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:56.733777 14571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:56.733810 14571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:56.733839 14571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:56.733857 14571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:56.739454 14571 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:32887
I20260806 15:03:56.739509 14743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:32887 every 8 connection(s)
I20260806 15:03:56.739796 14571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:03:56.745404 14544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:56.740478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:59243 (local address 127.0.236.126:42613)
0806 15:03:56.740557 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:56.740560 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:56.740741 (+ 181us) server_negotiation.cc:407] Connection header received
0806 15:03:56.741212 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:56.741213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:56.741225 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:56.741274 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:56.741870 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:56.742402 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:56.743705 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:56.743985 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:56.744062 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:56.745094 (+ 1032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:56.745096 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:56.745096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:03:56.745118 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:56.745135 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:03:56.745140 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:56.745193 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:56.745288 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 15:03:56.745319 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 15:03:56.745616 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 14571
I20260806 15:03:56.745743 14747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:56.740451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:42613 (local address 127.0.236.65:59243)
0806 15:03:56.740629 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:56.740661 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:56.741139 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:56.741328 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:56.741331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:56.741339 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:56.741810 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:56.741815 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:56.742504 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:56.742505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:56.743560 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:56.743563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:56.744072 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:56.744074 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:56.744233 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:56.744940 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:03:56.744941 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:56.744979 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:03:56.745007 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:56.745200 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:56.745205 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:56.745208 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:03:56.745288 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 15:03:56.745397 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260806 15:03:56.745774 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/wal/instance
I20260806 15:03:56.745994 945 external_mini_cluster.cc:1428] Setting key ac089c81770592c87052774856174785
I20260806 15:03:56.746522 14745 heartbeater.cc:347] Connected to a master server at 127.0.236.126:42613
I20260806 15:03:56.746632 14745 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:56.746811 14745 heartbeater.cc:511] Master 127.0.236.126:42613 requested a full tablet report, sending...
I20260806 15:03:56.746894 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:0
--local_ip_for_outbound_sockets=127.0.236.66
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:42613
--builtin_ntp_servers=127.0.236.84:39355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 15:03:56.747383 14478 ts_manager.cc:194] Registered new tserver with Master: 20564efbdaca418dab05790a436c8d02 (127.0.236.65:32887)
I20260806 15:03:56.747984 14478 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.236.65:59243
W20260806 15:03:56.826537 14748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:56.826725 14748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:56.826745 14748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:56.827854 14748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:56.827893 14748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:56.827920 14748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:03:56.829221 14748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:39355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:42613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:56.829435 14748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:56.829695 14748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:56.834299 14754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:56.834355 14757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:56.834312 14753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:56.834543 14748 server_base.cc:1034] running on GCE node
I20260806 15:03:56.834758 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:56.835003 14748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:56.836136 14748 hybrid_clock.cc:648] HybridClock initialized: now 1786028636836120 us; error 28 us; skew 500 ppm
I20260806 15:03:56.837255 14748 webserver.cc:460] Webserver started at http://127.0.236.66:36865/ using document root <none> and password file <none>
I20260806 15:03:56.837486 14748 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:56.837535 14748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:56.837661 14748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:56.838670 14748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/data/instance:
uuid: "130e7dde6a2846f99c942eb5f4484b00"
format_stamp: "Formatted at 2026-08-06 15:03:56 on dist-test-slave-tc2s"
server_key: "57775f0c664f05408173925312fca7c1"
server_key_iv: "f3d7019983009c2938c68f0dd5791487"
server_key_version: "encryptionkey@0"
I20260806 15:03:56.838984 14748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/wal/instance:
uuid: "130e7dde6a2846f99c942eb5f4484b00"
format_stamp: "Formatted at 2026-08-06 15:03:56 on dist-test-slave-tc2s"
server_key: "57775f0c664f05408173925312fca7c1"
server_key_iv: "f3d7019983009c2938c68f0dd5791487"
server_key_version: "encryptionkey@0"
I20260806 15:03:56.840142 14748 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 15:03:56.840806 14765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:56.840956 14748 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:03:56.841035 14748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "130e7dde6a2846f99c942eb5f4484b00"
format_stamp: "Formatted at 2026-08-06 15:03:56 on dist-test-slave-tc2s"
server_key: "57775f0c664f05408173925312fca7c1"
server_key_iv: "f3d7019983009c2938c68f0dd5791487"
server_key_version: "encryptionkey@0"
I20260806 15:03:56.841099 14748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:56.854095 14748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:56.854317 14748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:56.854431 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:56.854611 14748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:56.854882 14748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:56.854914 14748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:56.854939 14748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:56.854957 14748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:56.860054 14748 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:42501
I20260806 15:03:56.860112 14939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:42501 every 8 connection(s)
I20260806 15:03:56.860381 14748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
I20260806 15:03:56.862078 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 14748
I20260806 15:03:56.862162 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/wal/instance
I20260806 15:03:56.862330 945 external_mini_cluster.cc:1428] Setting key 7d5d75264c652f6aab59b87938d68deb
I20260806 15:03:56.863226 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:0
--local_ip_for_outbound_sockets=127.0.236.67
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:42613
--builtin_ntp_servers=127.0.236.84:39355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 15:03:56.865335 14544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:56.861005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:38583 (local address 127.0.236.126:42613)
0806 15:03:56.861059 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:56.861061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:56.861267 (+ 206us) server_negotiation.cc:407] Connection header received
0806 15:03:56.861692 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:56.861693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:56.861701 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:56.861750 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:56.862455 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:56.863101 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:56.864068 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:56.864320 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:56.864383 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:56.865086 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:56.865087 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:56.865088 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:03:56.865092 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:56.865109 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:03:56.865113 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:56.865159 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:56.865230 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 15:03:56.865258 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 15:03:56.865518 14942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:56.860972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:42613 (local address 127.0.236.66:38583)
0806 15:03:56.861159 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:56.861197 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:56.861621 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:56.861797 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:56.861800 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:56.861808 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:56.862369 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:56.862374 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:56.863183 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:56.863184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:56.863983 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:56.863985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:56.864393 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:56.864394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:56.864496 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:56.864959 (+ 463us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:03:56.864960 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:56.864986 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:03:56.865007 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:56.865166 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:56.865172 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:56.865174 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 15:03:56.865233 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 15:03:56.865331 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260806 15:03:56.865944 14940 heartbeater.cc:347] Connected to a master server at 127.0.236.126:42613
I20260806 15:03:56.866039 14940 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:56.866199 14940 heartbeater.cc:511] Master 127.0.236.126:42613 requested a full tablet report, sending...
I20260806 15:03:56.866555 14478 ts_manager.cc:194] Registered new tserver with Master: 130e7dde6a2846f99c942eb5f4484b00 (127.0.236.66:42501)
I20260806 15:03:56.866994 14478 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.236.66:38583
W20260806 15:03:56.942584 14943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:56.942759 14943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:56.942778 14943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:56.943876 14943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:56.943918 14943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:56.943943 14943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:03:56.945277 14943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:39355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:42613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:56.945451 14943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:56.945679 14943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:56.950240 14954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:56.950253 14950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:56.950446 14943 server_base.cc:1034] running on GCE node
W20260806 15:03:56.950241 14949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:56.950687 14943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:56.950904 14943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:56.952049 14943 hybrid_clock.cc:648] HybridClock initialized: now 1786028636952037 us; error 24 us; skew 500 ppm
I20260806 15:03:56.953187 14943 webserver.cc:460] Webserver started at http://127.0.236.67:40125/ using document root <none> and password file <none>
I20260806 15:03:56.953403 14943 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:56.953454 14943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:56.953573 14943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:56.954566 14943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/data/instance:
uuid: "1515f8a1753d4a3d83fd8f5c076cb4d2"
format_stamp: "Formatted at 2026-08-06 15:03:56 on dist-test-slave-tc2s"
server_key: "15a01dab65a2ba255bb32e5b52ac21b9"
server_key_iv: "856975876411638a36068bc515e249c5"
server_key_version: "encryptionkey@0"
I20260806 15:03:56.954865 14943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/wal/instance:
uuid: "1515f8a1753d4a3d83fd8f5c076cb4d2"
format_stamp: "Formatted at 2026-08-06 15:03:56 on dist-test-slave-tc2s"
server_key: "15a01dab65a2ba255bb32e5b52ac21b9"
server_key_iv: "856975876411638a36068bc515e249c5"
server_key_version: "encryptionkey@0"
I20260806 15:03:56.955955 14943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:03:56.956537 14963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:56.956681 14943 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:56.956758 14943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "1515f8a1753d4a3d83fd8f5c076cb4d2"
format_stamp: "Formatted at 2026-08-06 15:03:56 on dist-test-slave-tc2s"
server_key: "15a01dab65a2ba255bb32e5b52ac21b9"
server_key_iv: "856975876411638a36068bc515e249c5"
server_key_version: "encryptionkey@0"
I20260806 15:03:56.956815 14943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:56.975922 14943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:56.976158 14943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:56.976271 14943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:56.976475 14943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:56.976768 14943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:56.976800 14943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:56.976827 14943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:56.976843 14943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:56.982129 14943 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:40793
I20260806 15:03:56.982182 15130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:40793 every 8 connection(s)
I20260806 15:03:56.982452 14943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
I20260806 15:03:56.987195 14544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:56.983081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:58891 (local address 127.0.236.126:42613)
0806 15:03:56.983139 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:56.983141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:56.983357 (+ 216us) server_negotiation.cc:407] Connection header received
0806 15:03:56.983809 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:56.983811 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:56.983823 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:56.983870 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:56.984495 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:56.984990 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:56.985939 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:56.986142 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:56.986196 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:56.986928 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:56.986929 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:56.986930 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:03:56.986934 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:56.986951 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:03:56.986955 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:56.987003 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:56.987081 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 15:03:56.987115 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 15:03:56.987365 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:56.983062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:42613 (local address 127.0.236.67:58891)
0806 15:03:56.983254 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:56.983284 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:56.983721 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:56.983945 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:56.983949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:56.983956 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:56.984413 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:56.984418 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:56.985067 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:56.985068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:56.985830 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:56.985832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:56.986208 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:56.986209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:56.986331 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:56.986794 (+ 463us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:03:56.986795 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:56.986822 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:03:56.986842 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:56.987008 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:56.987012 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:56.987015 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:03:56.987082 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 15:03:56.987180 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":93,"threads_started":1}
I20260806 15:03:56.987761 15131 heartbeater.cc:347] Connected to a master server at 127.0.236.126:42613
I20260806 15:03:56.987841 15131 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:56.988013 15131 heartbeater.cc:511] Master 127.0.236.126:42613 requested a full tablet report, sending...
I20260806 15:03:56.988343 14478 ts_manager.cc:194] Registered new tserver with Master: 1515f8a1753d4a3d83fd8f5c076cb4d2 (127.0.236.67:40793)
I20260806 15:03:56.988694 14478 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.236.67:58891
I20260806 15:03:56.988786 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 14943
I20260806 15:03:56.988883 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/wal/instance
I20260806 15:03:56.989084 945 external_mini_cluster.cc:1428] Setting key 3f8a37814f88900f7199047178860b93
I20260806 15:03:56.989883 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 15:03:56.991511 14544 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:56.991103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36190 (local address 127.0.236.126:42613)
0806 15:03:56.991161 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:56.991163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:56.991261 (+ 98us) server_negotiation.cc:407] Connection header received
0806 15:03:56.991298 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:56.991299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:56.991308 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:56.991353 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:56.991454 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36190: BlockingRecv error: recv got EOF from 127.0.0.1:36190 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 15:03:56.991613 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 14571
I20260806 15:03:57.022667 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 14748
I20260806 15:03:57.034349 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 14943
I20260806 15:03:57.061278 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 14427
2026-08-06T15:03:57Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (653 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T15:03:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:03:57Z Disabled control of system clock
I20260806 15:03:57.091074 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:44255
--webserver_interface=127.0.236.126
--webserver_port=0
--builtin_ntp_servers=127.0.236.84:46077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:44255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 15:03:57.169626 15150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:57.169814 15150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:57.169834 15150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:57.170955 15150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:03:57.170994 15150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:57.171005 15150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:57.171016 15150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:03:57.171038 15150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:03:57.172304 15150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:46077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:44255
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:44255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:57.172484 15150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:57.172704 15150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:57.177274 15158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:57.177286 15160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:57.177300 15155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:57.177457 15150 server_base.cc:1034] running on GCE node
I20260806 15:03:57.177695 15150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:57.178002 15150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:57.179137 15150 hybrid_clock.cc:648] HybridClock initialized: now 1786028637179125 us; error 30 us; skew 500 ppm
I20260806 15:03:57.180236 15150 webserver.cc:460] Webserver started at http://127.0.236.126:35783/ using document root <none> and password file <none>
I20260806 15:03:57.180466 15150 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:57.180516 15150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:57.180644 15150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:57.181630 15150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "c52fde094026414b84ee5d3412c047eb"
format_stamp: "Formatted at 2026-08-06 15:03:57 on dist-test-slave-tc2s"
server_key: "0cc70640b834ea500ae3198e3ba0b735"
server_key_iv: "c5050f23196de986a89a55724cde6fcb"
server_key_version: "encryptionkey@0"
I20260806 15:03:57.182010 15150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "c52fde094026414b84ee5d3412c047eb"
format_stamp: "Formatted at 2026-08-06 15:03:57 on dist-test-slave-tc2s"
server_key: "0cc70640b834ea500ae3198e3ba0b735"
server_key_iv: "c5050f23196de986a89a55724cde6fcb"
server_key_version: "encryptionkey@0"
I20260806 15:03:57.183161 15150 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:03:57.183813 15168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:57.183964 15150 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:03:57.184054 15150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "c52fde094026414b84ee5d3412c047eb"
format_stamp: "Formatted at 2026-08-06 15:03:57 on dist-test-slave-tc2s"
server_key: "0cc70640b834ea500ae3198e3ba0b735"
server_key_iv: "c5050f23196de986a89a55724cde6fcb"
server_key_version: "encryptionkey@0"
I20260806 15:03:57.184118 15150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:57.187183 15150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:57.187381 15150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:57.187472 15150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:57.191035 15150 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:44255
I20260806 15:03:57.191064 15242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:44255 every 8 connection(s)
I20260806 15:03:57.191331 15150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:03:57.191998 15245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:57.194734 15245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c52fde094026414b84ee5d3412c047eb: Bootstrap starting.
I20260806 15:03:57.194739 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 15150
I20260806 15:03:57.194859 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:03:57.195060 945 external_mini_cluster.cc:1428] Setting key 26ed2c6a921ec07a20c933a4118a9d1f
I20260806 15:03:57.195423 15245 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c52fde094026414b84ee5d3412c047eb: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:57.195760 15245 log.cc:824] T 00000000000000000000000000000000 P c52fde094026414b84ee5d3412c047eb: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:57.196470 15245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c52fde094026414b84ee5d3412c047eb: No bootstrap required, opened a new log
I20260806 15:03:57.197374 15249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:57.195530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57684 (local address 127.0.236.126:44255)
0806 15:03:57.195703 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:57.195710 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:57.195733 (+ 23us) server_negotiation.cc:407] Connection header received
0806 15:03:57.196202 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:57.196211 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:57.196255 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:57.196365 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:57.196979 (+ 614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:57.196982 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:57.196982 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:03:57.196985 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:57.197020 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:03:57.197029 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:57.197093 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:57.197155 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 15:03:57.197199 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":67,"threads_started":1}
I20260806 15:03:57.198413 15245 raft_consensus.cc:348] T 00000000000000000000000000000000 P c52fde094026414b84ee5d3412c047eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c52fde094026414b84ee5d3412c047eb" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44255 } }
I20260806 15:03:57.198536 15245 raft_consensus.cc:374] T 00000000000000000000000000000000 P c52fde094026414b84ee5d3412c047eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:57.198557 15245 raft_consensus.cc:729] T 00000000000000000000000000000000 P c52fde094026414b84ee5d3412c047eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c52fde094026414b84ee5d3412c047eb, State: Initialized, Role: FOLLOWER
I20260806 15:03:57.198644 15245 consensus_queue.cc:260] T 00000000000000000000000000000000 P c52fde094026414b84ee5d3412c047eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c52fde094026414b84ee5d3412c047eb" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44255 } }
I20260806 15:03:57.198724 15245 raft_consensus.cc:388] T 00000000000000000000000000000000 P c52fde094026414b84ee5d3412c047eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:03:57.198750 15245 raft_consensus.cc:482] T 00000000000000000000000000000000 P c52fde094026414b84ee5d3412c047eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:03:57.198781 15245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c52fde094026414b84ee5d3412c047eb [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:57.199502 15245 raft_consensus.cc:504] T 00000000000000000000000000000000 P c52fde094026414b84ee5d3412c047eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c52fde094026414b84ee5d3412c047eb" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44255 } }
I20260806 15:03:57.199589 15245 leader_election.cc:302] T 00000000000000000000000000000000 P c52fde094026414b84ee5d3412c047eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c52fde094026414b84ee5d3412c047eb; no voters:
I20260806 15:03:57.199699 15245 leader_election.cc:288] T 00000000000000000000000000000000 P c52fde094026414b84ee5d3412c047eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:03:57.199747 15251 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c52fde094026414b84ee5d3412c047eb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:57.199935 15251 raft_consensus.cc:686] T 00000000000000000000000000000000 P c52fde094026414b84ee5d3412c047eb [term 1 LEADER]: Becoming Leader. State: Replica: c52fde094026414b84ee5d3412c047eb, State: Running, Role: LEADER
I20260806 15:03:57.199946 15245 sys_catalog.cc:564] T 00000000000000000000000000000000 P c52fde094026414b84ee5d3412c047eb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:03:57.200037 15251 consensus_queue.cc:237] T 00000000000000000000000000000000 P c52fde094026414b84ee5d3412c047eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c52fde094026414b84ee5d3412c047eb" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44255 } }
I20260806 15:03:57.200381 15253 sys_catalog.cc:455] T 00000000000000000000000000000000 P c52fde094026414b84ee5d3412c047eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c52fde094026414b84ee5d3412c047eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c52fde094026414b84ee5d3412c047eb" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44255 } } }
I20260806 15:03:57.200505 15253 sys_catalog.cc:458] T 00000000000000000000000000000000 P c52fde094026414b84ee5d3412c047eb [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:57.200385 15254 sys_catalog.cc:455] T 00000000000000000000000000000000 P c52fde094026414b84ee5d3412c047eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader c52fde094026414b84ee5d3412c047eb. Latest consensus state: current_term: 1 leader_uuid: "c52fde094026414b84ee5d3412c047eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c52fde094026414b84ee5d3412c047eb" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44255 } } }
I20260806 15:03:57.200685 15254 sys_catalog.cc:458] T 00000000000000000000000000000000 P c52fde094026414b84ee5d3412c047eb [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:57.200954 15268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:03:57.201385 15268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:03:57.202713 15268 catalog_manager.cc:1347] Generated new cluster ID: 1425be4ac0cb4837960149f2bf54e120
I20260806 15:03:57.202765 15268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:03:57.216645 15268 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:03:57.216787 15268 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:03:57.219906 15268 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c52fde094026414b84ee5d3412c047eb: Generated new TSK 0
I20260806 15:03:57.220059 15268 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:03:57.227175 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:0
--local_ip_for_outbound_sockets=127.0.236.65
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44255
--builtin_ntp_servers=127.0.236.84:46077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 15:03:57.306680 15281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:57.306859 15281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:57.306879 15281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:57.307984 15281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:57.308020 15281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:57.308044 15281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:03:57.309334 15281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:46077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:57.309551 15281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:57.309814 15281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:57.314445 15288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:57.314471 15292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:57.314458 15289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:57.314875 15281 server_base.cc:1034] running on GCE node
I20260806 15:03:57.315066 15281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:57.315274 15281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:57.316411 15281 hybrid_clock.cc:648] HybridClock initialized: now 1786028637316398 us; error 29 us; skew 500 ppm
I20260806 15:03:57.317577 15281 webserver.cc:460] Webserver started at http://127.0.236.65:37843/ using document root <none> and password file <none>
I20260806 15:03:57.317795 15281 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:57.317842 15281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:57.317991 15281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:57.318953 15281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/data/instance:
uuid: "a53d5ef9c723476eb015754934f10bc8"
format_stamp: "Formatted at 2026-08-06 15:03:57 on dist-test-slave-tc2s"
server_key: "fa9f04a08fea8878e7bed407f4226a16"
server_key_iv: "521a0d4dac722a9906ea2f389b4de6d9"
server_key_version: "encryptionkey@0"
I20260806 15:03:57.319269 15281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/wal/instance:
uuid: "a53d5ef9c723476eb015754934f10bc8"
format_stamp: "Formatted at 2026-08-06 15:03:57 on dist-test-slave-tc2s"
server_key: "fa9f04a08fea8878e7bed407f4226a16"
server_key_iv: "521a0d4dac722a9906ea2f389b4de6d9"
server_key_version: "encryptionkey@0"
I20260806 15:03:57.320502 15281 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:03:57.321197 15299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:57.321347 15281 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:57.321427 15281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "a53d5ef9c723476eb015754934f10bc8"
format_stamp: "Formatted at 2026-08-06 15:03:57 on dist-test-slave-tc2s"
server_key: "fa9f04a08fea8878e7bed407f4226a16"
server_key_iv: "521a0d4dac722a9906ea2f389b4de6d9"
server_key_version: "encryptionkey@0"
I20260806 15:03:57.321496 15281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:57.324637 15281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:57.324877 15281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:57.324990 15281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:57.325178 15281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:57.325448 15281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:57.325480 15281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:57.325506 15281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:57.325529 15281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:57.330806 15281 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:40951
I20260806 15:03:57.330869 15461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:40951 every 8 connection(s)
I20260806 15:03:57.331137 15281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:03:57.332544 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 15281
I20260806 15:03:57.332618 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-0/wal/instance
I20260806 15:03:57.332782 945 external_mini_cluster.cc:1428] Setting key d0b52e8aa5c0a252cd94fe2dde08403c
I20260806 15:03:57.333727 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:0
--local_ip_for_outbound_sockets=127.0.236.66
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44255
--builtin_ntp_servers=127.0.236.84:46077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 15:03:57.334103 15249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:57.331744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:47359 (local address 127.0.236.126:44255)
0806 15:03:57.331800 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:57.331802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:57.331994 (+ 192us) server_negotiation.cc:407] Connection header received
0806 15:03:57.332436 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:57.332437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:57.332444 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:57.332492 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:57.333756 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:57.333757 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:57.333758 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:03:57.333762 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:57.333777 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:03:57.333782 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:57.333832 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:57.334019 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 15:03:57.334034 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 15:03:57.334313 15466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:57.331717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:44255 (local address 127.0.236.65:47359)
0806 15:03:57.331895 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:57.331924 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:57.332346 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:57.332574 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:57.332578 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:57.332587 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:57.333614 (+ 1027us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:03:57.333614 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:57.333642 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:03:57.333668 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:57.333930 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:57.333943 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:57.333947 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:03:57.334021 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 15:03:57.334082 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
I20260806 15:03:57.334721 15463 heartbeater.cc:347] Connected to a master server at 127.0.236.126:44255
I20260806 15:03:57.334803 15463 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:57.334944 15463 heartbeater.cc:511] Master 127.0.236.126:44255 requested a full tablet report, sending...
I20260806 15:03:57.335386 15188 ts_manager.cc:194] Registered new tserver with Master: a53d5ef9c723476eb015754934f10bc8 (127.0.236.65:40951)
W20260806 15:03:57.415870 15467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:57.416080 15467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:57.416101 15467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:57.417268 15467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:57.417307 15467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:57.417335 15467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:03:57.418752 15467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:46077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:57.418988 15467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:57.419275 15467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:57.423952 15475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:57.423950 15472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:57.423974 15473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:57.424187 15467 server_base.cc:1034] running on GCE node
I20260806 15:03:57.424417 15467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:57.424640 15467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:57.425791 15467 hybrid_clock.cc:648] HybridClock initialized: now 1786028637425769 us; error 35 us; skew 500 ppm
I20260806 15:03:57.427038 15467 webserver.cc:460] Webserver started at http://127.0.236.66:43115/ using document root <none> and password file <none>
I20260806 15:03:57.427264 15467 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:57.427313 15467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:57.427460 15467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:57.428496 15467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/data/instance:
uuid: "a12785e0e7d74872adf5cb66640d7794"
format_stamp: "Formatted at 2026-08-06 15:03:57 on dist-test-slave-tc2s"
server_key: "bf06a34c0b6cffb8617e4cbc80869ef4"
server_key_iv: "995fbcd72b53eca1908eb61d4c4bb6d1"
server_key_version: "encryptionkey@0"
I20260806 15:03:57.428812 15467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/wal/instance:
uuid: "a12785e0e7d74872adf5cb66640d7794"
format_stamp: "Formatted at 2026-08-06 15:03:57 on dist-test-slave-tc2s"
server_key: "bf06a34c0b6cffb8617e4cbc80869ef4"
server_key_iv: "995fbcd72b53eca1908eb61d4c4bb6d1"
server_key_version: "encryptionkey@0"
I20260806 15:03:57.430012 15467 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:03:57.430647 15482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:57.430792 15467 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:57.430874 15467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "a12785e0e7d74872adf5cb66640d7794"
format_stamp: "Formatted at 2026-08-06 15:03:57 on dist-test-slave-tc2s"
server_key: "bf06a34c0b6cffb8617e4cbc80869ef4"
server_key_iv: "995fbcd72b53eca1908eb61d4c4bb6d1"
server_key_version: "encryptionkey@0"
I20260806 15:03:57.430941 15467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:57.434113 15467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:57.434336 15467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:57.434460 15467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:57.434640 15467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:57.434913 15467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:57.434947 15467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:57.434973 15467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:57.434996 15467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:57.440285 15467 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:40089
I20260806 15:03:57.440330 15634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:40089 every 8 connection(s)
I20260806 15:03:57.440618 15467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
I20260806 15:03:57.443203 15249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:57.441213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:43579 (local address 127.0.236.126:44255)
0806 15:03:57.441284 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:57.441286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:57.441462 (+ 176us) server_negotiation.cc:407] Connection header received
0806 15:03:57.441877 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:57.441878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:57.441883 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:57.441958 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:57.443033 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:57.443034 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:57.443034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:03:57.443037 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:57.443046 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:03:57.443050 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:57.443079 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:57.443134 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 15:03:57.443145 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 15:03:57.443365 15640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:57.441195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:44255 (local address 127.0.236.66:43579)
0806 15:03:57.441354 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:57.441385 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:57.441793 (+ 408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:57.442020 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:57.442024 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:57.442031 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:57.442910 (+ 879us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:03:57.442911 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:57.442937 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:03:57.442964 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:57.443087 (+ 123us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:57.443089 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:57.443091 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 15:03:57.443137 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 15:03:57.443182 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":79,"threads_started":1}
I20260806 15:03:57.443813 15636 heartbeater.cc:347] Connected to a master server at 127.0.236.126:44255
I20260806 15:03:57.443898 15636 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:57.444011 15636 heartbeater.cc:511] Master 127.0.236.126:44255 requested a full tablet report, sending...
I20260806 15:03:57.444316 15188 ts_manager.cc:194] Registered new tserver with Master: a12785e0e7d74872adf5cb66640d7794 (127.0.236.66:40089)
I20260806 15:03:57.448707 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 15467
I20260806 15:03:57.448786 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-1/wal/instance
I20260806 15:03:57.448947 945 external_mini_cluster.cc:1428] Setting key 952c89662146d5924b546696aaacb4de
I20260806 15:03:57.449723 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:0
--local_ip_for_outbound_sockets=127.0.236.67
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44255
--builtin_ntp_servers=127.0.236.84:46077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 15:03:57.529273 15641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:57.529471 15641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:57.529500 15641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:57.530659 15641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:57.530699 15641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:57.530731 15641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:03:57.532013 15641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:46077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:57.532229 15641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:57.532485 15641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:57.537204 15650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:57.537210 15648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:57.537272 15641 server_base.cc:1034] running on GCE node
W20260806 15:03:57.537273 15647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:57.537623 15641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:57.537815 15641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:57.538945 15641 hybrid_clock.cc:648] HybridClock initialized: now 1786028637538946 us; error 43 us; skew 500 ppm
I20260806 15:03:57.540126 15641 webserver.cc:460] Webserver started at http://127.0.236.67:46867/ using document root <none> and password file <none>
I20260806 15:03:57.540330 15641 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:57.540375 15641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:57.540508 15641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:57.541474 15641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/data/instance:
uuid: "000166680913413495c72b4bb868549f"
format_stamp: "Formatted at 2026-08-06 15:03:57 on dist-test-slave-tc2s"
server_key: "c1ac8be2460655d3d6331a2ce5278b4f"
server_key_iv: "f10c3da18f60f112f9d0c9d5a327a38c"
server_key_version: "encryptionkey@0"
I20260806 15:03:57.541779 15641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/wal/instance:
uuid: "000166680913413495c72b4bb868549f"
format_stamp: "Formatted at 2026-08-06 15:03:57 on dist-test-slave-tc2s"
server_key: "c1ac8be2460655d3d6331a2ce5278b4f"
server_key_iv: "f10c3da18f60f112f9d0c9d5a327a38c"
server_key_version: "encryptionkey@0"
I20260806 15:03:57.542930 15641 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:03:57.543529 15656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:57.543676 15641 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:03:57.543756 15641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "000166680913413495c72b4bb868549f"
format_stamp: "Formatted at 2026-08-06 15:03:57 on dist-test-slave-tc2s"
server_key: "c1ac8be2460655d3d6331a2ce5278b4f"
server_key_iv: "f10c3da18f60f112f9d0c9d5a327a38c"
server_key_version: "encryptionkey@0"
I20260806 15:03:57.543814 15641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:57.546994 15641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:57.547211 15641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:57.547322 15641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:57.547505 15641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:57.547768 15641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:57.547801 15641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:57.547825 15641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:57.547849 15641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:57.553057 15641 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:46799
I20260806 15:03:57.553093 15816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:46799 every 8 connection(s)
I20260806 15:03:57.553359 15641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
I20260806 15:03:57.554294 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 15641
I20260806 15:03:57.554385 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028580791223-945-0/minicluster-data/ts-2/wal/instance
I20260806 15:03:57.554546 945 external_mini_cluster.cc:1428] Setting key eb86a1c86c2c7ff9fc193006cf0da165
I20260806 15:03:57.556052 15249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:57.553996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:44889 (local address 127.0.236.126:44255)
0806 15:03:57.554044 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:57.554045 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:57.554278 (+ 233us) server_negotiation.cc:407] Connection header received
0806 15:03:57.554744 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:57.554745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:57.554752 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:57.554798 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:57.555878 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:57.555879 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:57.555879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:03:57.555884 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:57.555898 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:03:57.555902 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:57.555944 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:57.555991 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 15:03:57.556002 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 15:03:57.556212 15821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:57.554000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:44255 (local address 127.0.236.67:44889)
0806 15:03:57.554173 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:57.554202 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:57.554633 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:57.554861 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:57.554865 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:57.554875 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:57.555753 (+ 878us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:03:57.555753 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:57.555779 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:03:57.555804 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:57.555947 (+ 143us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:57.555949 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:57.555951 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 15:03:57.555996 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 15:03:57.556040 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":76,"threads_started":1}
I20260806 15:03:57.556612 15818 heartbeater.cc:347] Connected to a master server at 127.0.236.126:44255
I20260806 15:03:57.556694 15818 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:57.556826 15818 heartbeater.cc:511] Master 127.0.236.126:44255 requested a full tablet report, sending...
I20260806 15:03:57.557140 15188 ts_manager.cc:194] Registered new tserver with Master: 000166680913413495c72b4bb868549f (127.0.236.67:46799)
I20260806 15:03:57.565989 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 15:03:57.567574 15249 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:03:57.567196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57688 (local address 127.0.236.126:44255)
0806 15:03:57.567246 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:57.567248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:57.567331 (+ 83us) server_negotiation.cc:407] Connection header received
0806 15:03:57.567383 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:57.567384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:57.567388 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:57.567431 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:57.567520 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57688: BlockingRecv error: recv got EOF from 127.0.0.1:57688 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 15:03:57.567673 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 15281
I20260806 15:03:57.578104 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 15467
I20260806 15:03:57.596992 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 15641
I20260806 15:03:57.609584 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 15150
2026-08-06T15:03:57Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (547 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:03:57 dist-test-slave-tc2s krb5kdc[15830](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:03:57 dist-test-slave-tc2s krb5kdc[15830](info): set up 2 sockets
Aug 06 15:03:57 dist-test-slave-tc2s krb5kdc[15830](info): commencing operation
krb5kdc: starting...
W20260806 15:03:58.724086 945 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:03:58 dist-test-slave-tc2s krb5kdc[15830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:03:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:03:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:03:58.835813 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:37031
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:41639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:37031
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:58.914628 15849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:58.914814 15849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:58.914836 15849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:58.915951 15849 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:03:58.915989 15849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:58.916002 15849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:58.916025 15849 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:03:58.916044 15849 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:03:58.917482 15849 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:41639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:37031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:37031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:58.917702 15849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:58.917913 15849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:58.922390 15858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:58.922405 15862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:58.922405 15859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:58.922622 15849 server_base.cc:1034] running on GCE node
I20260806 15:03:58.922857 15849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:58.923127 15849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:58.924261 15849 hybrid_clock.cc:648] HybridClock initialized: now 1786028638924246 us; error 33 us; skew 500 ppm
Aug 06 15:03:58 dist-test-slave-tc2s krb5kdc[15830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028638, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:58.926575 15849 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:03:58.926983 15849 webserver.cc:460] Webserver started at http://127.0.236.126:37035/ using document root <none> and password file <none>
I20260806 15:03:58.927170 15849 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:58.927217 15849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:58.927345 15849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:58.928303 15849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "251312685abd4500820af23bee9689cb"
format_stamp: "Formatted at 2026-08-06 15:03:58 on dist-test-slave-tc2s"
server_key: "853985f67eba6ca6b904a158bf312294"
server_key_iv: "06c35151afaa308f8a3afbee0da4069e"
server_key_version: "encryptionkey@0"
I20260806 15:03:58.928633 15849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "251312685abd4500820af23bee9689cb"
format_stamp: "Formatted at 2026-08-06 15:03:58 on dist-test-slave-tc2s"
server_key: "853985f67eba6ca6b904a158bf312294"
server_key_iv: "06c35151afaa308f8a3afbee0da4069e"
server_key_version: "encryptionkey@0"
I20260806 15:03:58.929850 15849 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 15:03:58.930522 15874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:58.930670 15849 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:03:58.930752 15849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "251312685abd4500820af23bee9689cb"
format_stamp: "Formatted at 2026-08-06 15:03:58 on dist-test-slave-tc2s"
server_key: "853985f67eba6ca6b904a158bf312294"
server_key_iv: "06c35151afaa308f8a3afbee0da4069e"
server_key_version: "encryptionkey@0"
I20260806 15:03:58.930812 15849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:58.939839 15849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:58.941751 15849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:58.941877 15849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:58.945693 15849 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:37031
I20260806 15:03:58.945724 15943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:37031 every 8 connection(s)
I20260806 15:03:58.946051 15849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:03:58.946933 15944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:58.949011 15944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 251312685abd4500820af23bee9689cb: Bootstrap starting.
I20260806 15:03:58.949523 15944 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 251312685abd4500820af23bee9689cb: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:58.949746 15944 log.cc:824] T 00000000000000000000000000000000 P 251312685abd4500820af23bee9689cb: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:58.949859 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 15849
I20260806 15:03:58.949970 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:03:58.950142 945 external_mini_cluster.cc:1428] Setting key af13afdc5490468c932e8b72951b08be
I20260806 15:03:58.950497 15944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 251312685abd4500820af23bee9689cb: No bootstrap required, opened a new log
I20260806 15:03:58.952446 15944 raft_consensus.cc:348] T 00000000000000000000000000000000 P 251312685abd4500820af23bee9689cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "251312685abd4500820af23bee9689cb" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 37031 } }
I20260806 15:03:58.952613 15944 raft_consensus.cc:374] T 00000000000000000000000000000000 P 251312685abd4500820af23bee9689cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:58.952658 15944 raft_consensus.cc:729] T 00000000000000000000000000000000 P 251312685abd4500820af23bee9689cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 251312685abd4500820af23bee9689cb, State: Initialized, Role: FOLLOWER
I20260806 15:03:58.952786 15944 consensus_queue.cc:260] T 00000000000000000000000000000000 P 251312685abd4500820af23bee9689cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "251312685abd4500820af23bee9689cb" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 37031 } }
I20260806 15:03:58.952858 15944 raft_consensus.cc:388] T 00000000000000000000000000000000 P 251312685abd4500820af23bee9689cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:03:58.952893 15944 raft_consensus.cc:482] T 00000000000000000000000000000000 P 251312685abd4500820af23bee9689cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:03:58.952944 15944 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 251312685abd4500820af23bee9689cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:58.953576 15944 raft_consensus.cc:504] T 00000000000000000000000000000000 P 251312685abd4500820af23bee9689cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "251312685abd4500820af23bee9689cb" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 37031 } }
I20260806 15:03:58.953682 15944 leader_election.cc:302] T 00000000000000000000000000000000 P 251312685abd4500820af23bee9689cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 251312685abd4500820af23bee9689cb; no voters:
I20260806 15:03:58.953820 15944 leader_election.cc:288] T 00000000000000000000000000000000 P 251312685abd4500820af23bee9689cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:03:58.953866 15951 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 251312685abd4500820af23bee9689cb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:58.954072 15944 sys_catalog.cc:564] T 00000000000000000000000000000000 P 251312685abd4500820af23bee9689cb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:03:58.954095 15951 raft_consensus.cc:686] T 00000000000000000000000000000000 P 251312685abd4500820af23bee9689cb [term 1 LEADER]: Becoming Leader. State: Replica: 251312685abd4500820af23bee9689cb, State: Running, Role: LEADER
I20260806 15:03:58.954308 15951 consensus_queue.cc:237] T 00000000000000000000000000000000 P 251312685abd4500820af23bee9689cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "251312685abd4500820af23bee9689cb" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 37031 } }
I20260806 15:03:58.954674 15955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 251312685abd4500820af23bee9689cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 251312685abd4500820af23bee9689cb. Latest consensus state: current_term: 1 leader_uuid: "251312685abd4500820af23bee9689cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "251312685abd4500820af23bee9689cb" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 37031 } } }
I20260806 15:03:58.954689 15954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 251312685abd4500820af23bee9689cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "251312685abd4500820af23bee9689cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "251312685abd4500820af23bee9689cb" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 37031 } } }
I20260806 15:03:58.954867 15954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 251312685abd4500820af23bee9689cb [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:58.954807 15955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 251312685abd4500820af23bee9689cb [sys.catalog]: This master's current role is: LEADER
I20260806 15:03:58.955106 15968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:03:58.955675 15968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:03:58.957386 15968 catalog_manager.cc:1347] Generated new cluster ID: 0263c27979224296b0c48b6f17ad9a2e
I20260806 15:03:58.957430 15968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 15:03:58 dist-test-slave-tc2s krb5kdc[15830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:58.962293 15950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:58.950578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53142 (local address 127.0.236.126:37031)
0806 15:03:58.950760 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:58.950769 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:58.950800 (+ 31us) server_negotiation.cc:407] Connection header received
0806 15:03:58.951678 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:58.951688 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:58.951757 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:58.951901 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:58.952426 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:58.954066 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:58.955086 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:58.955691 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:58.955791 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:58.958510 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:58.958514 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:58.958515 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:58.958526 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:58.960310 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:58.960721 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:58.960723 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:58.960723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:58.960821 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:58.961008 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:58.961009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:58.961009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:58.961278 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:58.961870 (+ 592us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:58.961977 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 15:03:58.962111 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":95,"threads_started":1}
I20260806 15:03:58.993144 15968 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:03:58.993666 15968 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:03:58.998842 15968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 251312685abd4500820af23bee9689cb: Generated new TSK 0
I20260806 15:03:58.999002 15968 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.236.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
I20260806 15:03:59.028307 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:0
--local_ip_for_outbound_sockets=127.0.236.65
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:37031
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:41639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:59.107173 15982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:59.107365 15982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:59.107386 15982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:59.108491 15982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:59.108531 15982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:59.108559 15982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:03:59.109993 15982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:41639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:37031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:59.110200 15982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:59.110410 15982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:59.115029 15989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:59.115041 15991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:59.115015 15987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:59.115188 15982 server_base.cc:1034] running on GCE node
I20260806 15:03:59.115415 15982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:59.115646 15982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:59.116780 15982 hybrid_clock.cc:648] HybridClock initialized: now 1786028639116767 us; error 28 us; skew 500 ppm
Aug 06 15:03:59 dist-test-slave-tc2s krb5kdc[15830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028639, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:59.119107 15982 init.cc:377] Logged in from keytab as kudu/127.0.236.65@KRBTEST.COM (short username kudu)
I20260806 15:03:59.119542 15982 webserver.cc:460] Webserver started at http://127.0.236.65:42071/ using document root <none> and password file <none>
I20260806 15:03:59.119710 15982 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:59.119748 15982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:59.119837 15982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:59.120762 15982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-0/data/instance:
uuid: "904290f5ef46410791dab451856c9239"
format_stamp: "Formatted at 2026-08-06 15:03:59 on dist-test-slave-tc2s"
server_key: "3ae8914467a980513e244a690a144c60"
server_key_iv: "c87d5122bbe4886ce98341f50d50e76a"
server_key_version: "encryptionkey@0"
I20260806 15:03:59.121045 15982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-0/wal/instance:
uuid: "904290f5ef46410791dab451856c9239"
format_stamp: "Formatted at 2026-08-06 15:03:59 on dist-test-slave-tc2s"
server_key: "3ae8914467a980513e244a690a144c60"
server_key_iv: "c87d5122bbe4886ce98341f50d50e76a"
server_key_version: "encryptionkey@0"
I20260806 15:03:59.122177 15982 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 15:03:59.122808 16000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:59.122960 15982 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:03:59.123041 15982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "904290f5ef46410791dab451856c9239"
format_stamp: "Formatted at 2026-08-06 15:03:59 on dist-test-slave-tc2s"
server_key: "3ae8914467a980513e244a690a144c60"
server_key_iv: "c87d5122bbe4886ce98341f50d50e76a"
server_key_version: "encryptionkey@0"
I20260806 15:03:59.123106 15982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:59.150555 15982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:59.152136 15982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:59.152266 15982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:59.152464 15982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:59.152737 15982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:59.152770 15982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:59.152798 15982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:59.152815 15982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:59.158414 15982 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:38013
I20260806 15:03:59.158452 16160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:38013 every 8 connection(s)
I20260806 15:03:59.158730 15982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:03:59.162853 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 15982
I20260806 15:03:59.162968 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-0/wal/instance
I20260806 15:03:59.163146 945 external_mini_cluster.cc:1428] Setting key 10c2bb6e4d83aa7b140e6043203e664a
Aug 06 15:03:59 dist-test-slave-tc2s krb5kdc[15830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028639, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:59.167394 15950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:59.159349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:53969 (local address 127.0.236.126:37031)
0806 15:03:59.159400 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:59.159402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:59.159580 (+ 178us) server_negotiation.cc:407] Connection header received
0806 15:03:59.160035 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:59.160036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:59.160045 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:59.160090 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:59.160954 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.161441 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.162404 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.162600 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.162643 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:59.164838 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:59.164839 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:59.164840 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:59.164846 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:59.166150 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:59.166513 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:59.166514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:59.166515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:59.166575 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:59.166783 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:59.166784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:59.166784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:59.166889 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:59.166945 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:59.167202 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 15:03:59.167303 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 15:03:59.167515 16163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:59.159313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:37031 (local address 127.0.236.65:53969)
0806 15:03:59.159470 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:59.159510 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:59.159960 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:59.160146 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:59.160149 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:59.160711 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:59.160888 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:59.160892 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.161501 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.161502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:59.162347 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:59.162349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.162652 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.162652 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:59.162778 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:59.163242 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:59.163252 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:59.164751 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:59.166230 (+ 1479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:59.166236 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:59.166238 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:59.166451 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:59.166630 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:59.166631 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:59.166631 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:59.166731 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:59.166942 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:59.166948 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:59.167088 (+ 140us) client_negotiation.cc:769] Sending connection context
0806 15:03:59.167171 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 15:03:59.167313 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":82,"threads_started":1}
I20260806 15:03:59.168247 16161 heartbeater.cc:347] Connected to a master server at 127.0.236.126:37031
I20260806 15:03:59.168325 16161 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:59.168483 16161 heartbeater.cc:511] Master 127.0.236.126:37031 requested a full tablet report, sending...
I20260806 15:03:59.168972 15900 ts_manager.cc:194] Registered new tserver with Master: 904290f5ef46410791dab451856c9239 (127.0.236.65:38013)
I20260806 15:03:59.169549 15900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.65@KRBTEST.COM'} at 127.0.236.65:53969
WARNING: no policy specified for kudu/127.0.236.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
I20260806 15:03:59.186640 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:0
--local_ip_for_outbound_sockets=127.0.236.66
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:37031
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:41639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:59.264923 16166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:59.265110 16166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:59.265130 16166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:59.266233 16166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:59.266273 16166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:59.266297 16166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:03:59.267668 16166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:41639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:37031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:59.267834 16166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:59.268000 16166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:59.272634 16175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:59.272692 16166 server_base.cc:1034] running on GCE node
W20260806 15:03:59.272657 16173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:59.272657 16172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:59.273077 16166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:59.273268 16166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:59.274394 16166 hybrid_clock.cc:648] HybridClock initialized: now 1786028639274382 us; error 27 us; skew 500 ppm
Aug 06 15:03:59 dist-test-slave-tc2s krb5kdc[15830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028639, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:59.276671 16166 init.cc:377] Logged in from keytab as kudu/127.0.236.66@KRBTEST.COM (short username kudu)
I20260806 15:03:59.277076 16166 webserver.cc:460] Webserver started at http://127.0.236.66:39573/ using document root <none> and password file <none>
I20260806 15:03:59.277258 16166 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:59.277304 16166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:59.277427 16166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:59.278414 16166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-1/data/instance:
uuid: "cb72f47763f04546a5359cd3fb1dd2ca"
format_stamp: "Formatted at 2026-08-06 15:03:59 on dist-test-slave-tc2s"
server_key: "46c237595502c7b5ec1b7536bb144998"
server_key_iv: "d6b5ac36032726c2b5c46e9fb6ca238d"
server_key_version: "encryptionkey@0"
I20260806 15:03:59.278770 16166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-1/wal/instance:
uuid: "cb72f47763f04546a5359cd3fb1dd2ca"
format_stamp: "Formatted at 2026-08-06 15:03:59 on dist-test-slave-tc2s"
server_key: "46c237595502c7b5ec1b7536bb144998"
server_key_iv: "d6b5ac36032726c2b5c46e9fb6ca238d"
server_key_version: "encryptionkey@0"
I20260806 15:03:59.279978 16166 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:03:59.280648 16187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:59.280789 16166 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:03:59.280869 16166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "cb72f47763f04546a5359cd3fb1dd2ca"
format_stamp: "Formatted at 2026-08-06 15:03:59 on dist-test-slave-tc2s"
server_key: "46c237595502c7b5ec1b7536bb144998"
server_key_iv: "d6b5ac36032726c2b5c46e9fb6ca238d"
server_key_version: "encryptionkey@0"
I20260806 15:03:59.280934 16166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:59.291757 16166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:59.293278 16166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:59.293408 16166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:59.293602 16166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:59.293869 16166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:59.293933 16166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:59.293964 16166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:59.293987 16166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:59.299247 16166 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:35521
I20260806 15:03:59.299285 16347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:35521 every 8 connection(s)
I20260806 15:03:59.299563 16166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
I20260806 15:03:59.300581 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 16166
I20260806 15:03:59.300661 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-1/wal/instance
I20260806 15:03:59.300837 945 external_mini_cluster.cc:1428] Setting key 6ce81d737f28ed9fc6315f1c913e63b2
Aug 06 15:03:59 dist-test-slave-tc2s krb5kdc[15830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028639, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:59.308624 15950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:59.300194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:55711 (local address 127.0.236.126:37031)
0806 15:03:59.300235 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:59.300237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:59.300440 (+ 203us) server_negotiation.cc:407] Connection header received
0806 15:03:59.300913 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:59.300914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:59.300927 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:59.300981 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:59.301946 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.302443 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.303681 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.303880 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.303942 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:59.306120 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:59.306121 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:59.306121 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:59.306130 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:59.307389 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:59.307755 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:59.307756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:59.307756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:59.307811 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:59.308031 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:59.308032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:59.308032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:59.308141 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:59.308196 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:59.308450 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 15:03:59.308534 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:03:59.308777 16350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:59.300161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:37031 (local address 127.0.236.66:55711)
0806 15:03:59.300293 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:59.300338 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:59.300817 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:59.301045 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:59.301049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:59.301659 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:59.301836 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:59.301841 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.302524 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.302525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:59.303590 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:59.303594 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.303959 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.303960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:59.304102 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:59.304545 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:59.304554 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:59.306026 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:59.307470 (+ 1444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:59.307477 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:59.307479 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:59.307688 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:59.307866 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:59.307867 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:59.307867 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:59.307976 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:59.308197 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:59.308203 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:59.308334 (+ 131us) client_negotiation.cc:769] Sending connection context
0806 15:03:59.308420 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 15:03:59.308578 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":62,"threads_started":1}
WARNING: no policy specified for kudu/127.0.236.67@KRBTEST.COM; defaulting to no policy
I20260806 15:03:59.309213 16348 heartbeater.cc:347] Connected to a master server at 127.0.236.126:37031
I20260806 15:03:59.309314 16348 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:59.309492 16348 heartbeater.cc:511] Master 127.0.236.126:37031 requested a full tablet report, sending...
I20260806 15:03:59.309837 15900 ts_manager.cc:194] Registered new tserver with Master: cb72f47763f04546a5359cd3fb1dd2ca (127.0.236.66:35521)
I20260806 15:03:59.310240 15900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.66@KRBTEST.COM'} at 127.0.236.66:55711
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
I20260806 15:03:59.324286 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:0
--local_ip_for_outbound_sockets=127.0.236.67
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:37031
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:41639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:03:59.402702 16353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:03:59.402881 16353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:03:59.402904 16353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:03:59.404256 16353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:03:59.404305 16353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:03:59.404343 16353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:03:59.406150 16353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:41639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:37031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:03:59.406507 16353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:03:59.406761 16353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:03:59.415328 16359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:59.415465 16353 server_base.cc:1034] running on GCE node
W20260806 15:03:59.415513 16362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:03:59.415629 16364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:03:59.415886 16353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:03:59.416145 16353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:03:59.417299 16353 hybrid_clock.cc:648] HybridClock initialized: now 1786028639417277 us; error 36 us; skew 500 ppm
Aug 06 15:03:59 dist-test-slave-tc2s krb5kdc[15830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028639, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:03:59.419591 16353 init.cc:377] Logged in from keytab as kudu/127.0.236.67@KRBTEST.COM (short username kudu)
I20260806 15:03:59.419960 16353 webserver.cc:460] Webserver started at http://127.0.236.67:44069/ using document root <none> and password file <none>
I20260806 15:03:59.420146 16353 fs_manager.cc:359] Metadata directory not provided
I20260806 15:03:59.420184 16353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:03:59.420272 16353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:03:59.421213 16353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-2/data/instance:
uuid: "173df9b68d5a4e29b631ab8923aa4953"
format_stamp: "Formatted at 2026-08-06 15:03:59 on dist-test-slave-tc2s"
server_key: "c7fa383e0b67311bf718c2761e2d282f"
server_key_iv: "412378dccf963427caf0f90f106eb6e0"
server_key_version: "encryptionkey@0"
I20260806 15:03:59.421494 16353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-2/wal/instance:
uuid: "173df9b68d5a4e29b631ab8923aa4953"
format_stamp: "Formatted at 2026-08-06 15:03:59 on dist-test-slave-tc2s"
server_key: "c7fa383e0b67311bf718c2761e2d282f"
server_key_iv: "412378dccf963427caf0f90f106eb6e0"
server_key_version: "encryptionkey@0"
I20260806 15:03:59.422663 16353 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:03:59.423260 16375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:59.423398 16353 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:03:59.423473 16353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "173df9b68d5a4e29b631ab8923aa4953"
format_stamp: "Formatted at 2026-08-06 15:03:59 on dist-test-slave-tc2s"
server_key: "c7fa383e0b67311bf718c2761e2d282f"
server_key_iv: "412378dccf963427caf0f90f106eb6e0"
server_key_version: "encryptionkey@0"
I20260806 15:03:59.423537 16353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:03:59.433477 16353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:03:59.435439 16353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:03:59.435564 16353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:03:59.435752 16353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:03:59.436018 16353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:03:59.436051 16353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:59.436070 16353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:03:59.436094 16353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:03:59.441452 16353 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:39145
I20260806 15:03:59.441493 16534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:39145 every 8 connection(s)
I20260806 15:03:59.441766 16353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
Aug 06 15:03:59 dist-test-slave-tc2s krb5kdc[15830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028639, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:03:59.449522 15950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:59.442423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:55527 (local address 127.0.236.126:37031)
0806 15:03:59.442480 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:59.442482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:59.442718 (+ 236us) server_negotiation.cc:407] Connection header received
0806 15:03:59.443085 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:59.443086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:59.443095 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:59.443147 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:59.443985 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.444472 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.445365 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.445538 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.445580 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:59.447480 (+ 1900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:59.447481 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:59.447481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:59.447488 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:59.448524 (+ 1036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:59.448857 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:59.448857 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:59.448858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:59.448900 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:59.449080 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:59.449080 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:59.449081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:59.449172 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:59.449214 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:59.449404 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 15:03:59.449458 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 15:03:59.449703 16539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:59.442376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:37031 (local address 127.0.236.67:55527)
0806 15:03:59.442533 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:59.442572 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:59.443011 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:59.443198 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:59.443201 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:59.443736 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:03:59.443910 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:59.443915 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.444533 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.444534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:59.445304 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:59.445306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.445590 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.445590 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:59.445690 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:59.446111 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:03:59.446119 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:03:59.447392 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:03:59.448589 (+ 1197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:59.448595 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:59.448597 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:59.448803 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:59.448953 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:03:59.448953 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:03:59.448954 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:03:59.449040 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:03:59.449211 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:03:59.449214 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:03:59.449315 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 15:03:59.449387 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 15:03:59.449506 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260806 15:03:59.450163 16537 heartbeater.cc:347] Connected to a master server at 127.0.236.126:37031
I20260806 15:03:59.450253 16537 heartbeater.cc:464] Registering TS with master...
I20260806 15:03:59.450418 16537 heartbeater.cc:511] Master 127.0.236.126:37031 requested a full tablet report, sending...
I20260806 15:03:59.450727 15900 ts_manager.cc:194] Registered new tserver with Master: 173df9b68d5a4e29b631ab8923aa4953 (127.0.236.67:39145)
I20260806 15:03:59.451090 15900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.67@KRBTEST.COM'} at 127.0.236.67:55527
I20260806 15:03:59.451771 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 16353
I20260806 15:03:59.451869 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-2/wal/instance
I20260806 15:03:59.452033 945 external_mini_cluster.cc:1428] Setting key edd01214214d1b31dd32e85c34070205
I20260806 15:03:59.452736 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:03:59.458734 15950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:59.453859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53158 (local address 127.0.236.126:37031)
0806 15:03:59.453905 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:59.453907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:59.454021 (+ 114us) server_negotiation.cc:407] Connection header received
0806 15:03:59.454068 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:59.454069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:59.454077 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:59.454119 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:03:59.454615 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.455118 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.455796 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.455974 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.456026 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:59.456974 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:03:59.456975 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:03:59.456975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:03:59.456981 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:03:59.457965 (+ 984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:59.458228 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:59.458229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:59.458229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:59.458272 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:03:59.458377 (+ 105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:03:59.458378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:03:59.458378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:03:59.458486 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:03:59.458555 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:59.458630 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 15:03:59.458679 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260806 15:03:59.460026 15900 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:03:59.461493 15900 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:03:59.467279 16552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:59.463447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44084 (local address 127.0.236.66:35521)
0806 15:03:59.463629 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:59.463634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:59.463757 (+ 123us) server_negotiation.cc:407] Connection header received
0806 15:03:59.463803 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:59.463805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:59.463820 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:59.463885 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:59.464060 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.464860 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.466357 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.466922 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.466984 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:59.467041 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:59.467168 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 15:03:59.467207 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":99,"threads_started":1}
I20260806 15:03:59.467270 16547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:59.463354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:35521 (local address 127.0.0.1:44084)
0806 15:03:59.463634 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:59.463644 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:59.463701 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:59.463967 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:59.463967 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:59.463969 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:59.464007 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:59.464008 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.464964 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.464965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:59.466277 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:59.466279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.466995 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.466996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:59.467074 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:59.467087 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 15:03:59.467149 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 15:03:59.467199 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":1,"thread_start_us":63,"threads_started":1}
I20260806 15:03:59.467518 16548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:59.463351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:39145 (local address 127.0.0.1:51110)
0806 15:03:59.463509 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:59.463544 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:59.463609 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:59.463895 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:59.463898 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:59.463900 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:59.463958 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:59.463960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.465400 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.465401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:59.466476 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:59.466478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.467316 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.467317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:59.467387 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:59.467390 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:03:59.467419 (+ 29us) client_negotiation.cc:240] Negotiation successful
0806 15:03:59.467451 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":23,"thread_start_us":56,"threads_started":1}
I20260806 15:03:59.467772 16553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:59.463544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:38013 (local address 127.0.0.1:54492)
0806 15:03:59.463712 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:59.463721 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:59.463758 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:59.464121 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:59.464123 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:59.464125 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:59.464166 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:59.464168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.465415 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.465416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:59.466272 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:59.466275 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.467173 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.467175 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:59.467234 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:59.467238 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:03:59.467280 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 15:03:59.467711 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":2,"thread_start_us":40,"threads_started":1}
I20260806 15:03:59.467862 16550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:59.463415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51110 (local address 127.0.236.67:39145)
0806 15:03:59.463737 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:59.463741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:59.463750 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:03:59.463784 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:59.463786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:59.463801 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:59.463879 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:59.464027 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.465290 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.466550 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.467225 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.467674 (+ 449us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:59.467737 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:59.467761 (+ 24us) server_negotiation.cc:299] Negotiation successful
0806 15:03:59.467797 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":78,"threads_started":1}
I20260806 15:03:59.467989 16554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:59.463722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54492 (local address 127.0.236.65:38013)
0806 15:03:59.463914 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:59.463917 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:59.463927 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:03:59.463961 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:59.463963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:59.463976 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:59.464040 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:59.464241 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.465312 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.466356 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.467018 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.467086 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:59.467137 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:59.467871 (+ 734us) server_negotiation.cc:299] Negotiation successful
0806 15:03:59.467915 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":76,"threads_started":1}
I20260806 15:03:59.467921 16258 tablet_service.cc:1467] Processing CreateTablet for tablet 3bf9b68969b5401b801fe462d4b7d591 (DEFAULT_TABLE table=test-table [id=c642edff5e4c484db8abf2ca0ef34d0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:59.468261 16258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bf9b68969b5401b801fe462d4b7d591. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:59.468367 16447 tablet_service.cc:1467] Processing CreateTablet for tablet 3bf9b68969b5401b801fe462d4b7d591 (DEFAULT_TABLE table=test-table [id=c642edff5e4c484db8abf2ca0ef34d0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:59.468649 16447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bf9b68969b5401b801fe462d4b7d591. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:59.470607 16065 tablet_service.cc:1467] Processing CreateTablet for tablet 3bf9b68969b5401b801fe462d4b7d591 (DEFAULT_TABLE table=test-table [id=c642edff5e4c484db8abf2ca0ef34d0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:03:59.470965 16065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bf9b68969b5401b801fe462d4b7d591. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:03:59.471130 16556 tablet_bootstrap.cc:492] T 3bf9b68969b5401b801fe462d4b7d591 P cb72f47763f04546a5359cd3fb1dd2ca: Bootstrap starting.
I20260806 15:03:59.471668 16556 tablet_bootstrap.cc:654] T 3bf9b68969b5401b801fe462d4b7d591 P cb72f47763f04546a5359cd3fb1dd2ca: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:59.472103 16556 log.cc:824] T 3bf9b68969b5401b801fe462d4b7d591 P cb72f47763f04546a5359cd3fb1dd2ca: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:59.472404 16560 tablet_bootstrap.cc:492] T 3bf9b68969b5401b801fe462d4b7d591 P 173df9b68d5a4e29b631ab8923aa4953: Bootstrap starting.
I20260806 15:03:59.472834 16556 tablet_bootstrap.cc:492] T 3bf9b68969b5401b801fe462d4b7d591 P cb72f47763f04546a5359cd3fb1dd2ca: No bootstrap required, opened a new log
I20260806 15:03:59.472900 16556 ts_tablet_manager.cc:1397] T 3bf9b68969b5401b801fe462d4b7d591 P cb72f47763f04546a5359cd3fb1dd2ca: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:03:59.473202 16560 tablet_bootstrap.cc:654] T 3bf9b68969b5401b801fe462d4b7d591 P 173df9b68d5a4e29b631ab8923aa4953: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:59.473537 16563 tablet_bootstrap.cc:492] T 3bf9b68969b5401b801fe462d4b7d591 P 904290f5ef46410791dab451856c9239: Bootstrap starting.
I20260806 15:03:59.473757 16560 log.cc:824] T 3bf9b68969b5401b801fe462d4b7d591 P 173df9b68d5a4e29b631ab8923aa4953: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:59.474207 16563 tablet_bootstrap.cc:654] T 3bf9b68969b5401b801fe462d4b7d591 P 904290f5ef46410791dab451856c9239: Neither blocks nor log segments found. Creating new log.
I20260806 15:03:59.474550 16560 tablet_bootstrap.cc:492] T 3bf9b68969b5401b801fe462d4b7d591 P 173df9b68d5a4e29b631ab8923aa4953: No bootstrap required, opened a new log
I20260806 15:03:59.474620 16560 ts_tablet_manager.cc:1397] T 3bf9b68969b5401b801fe462d4b7d591 P 173df9b68d5a4e29b631ab8923aa4953: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260806 15:03:59.474694 16563 log.cc:824] T 3bf9b68969b5401b801fe462d4b7d591 P 904290f5ef46410791dab451856c9239: Log is configured to *not* fsync() on all Append() calls
I20260806 15:03:59.474774 16556 raft_consensus.cc:348] T 3bf9b68969b5401b801fe462d4b7d591 P cb72f47763f04546a5359cd3fb1dd2ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173df9b68d5a4e29b631ab8923aa4953" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 39145 } } peers { permanent_uuid: "cb72f47763f04546a5359cd3fb1dd2ca" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 35521 } } peers { permanent_uuid: "904290f5ef46410791dab451856c9239" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38013 } }
I20260806 15:03:59.474943 16556 raft_consensus.cc:374] T 3bf9b68969b5401b801fe462d4b7d591 P cb72f47763f04546a5359cd3fb1dd2ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:59.474984 16556 raft_consensus.cc:729] T 3bf9b68969b5401b801fe462d4b7d591 P cb72f47763f04546a5359cd3fb1dd2ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb72f47763f04546a5359cd3fb1dd2ca, State: Initialized, Role: FOLLOWER
I20260806 15:03:59.475093 16556 consensus_queue.cc:260] T 3bf9b68969b5401b801fe462d4b7d591 P cb72f47763f04546a5359cd3fb1dd2ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173df9b68d5a4e29b631ab8923aa4953" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 39145 } } peers { permanent_uuid: "cb72f47763f04546a5359cd3fb1dd2ca" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 35521 } } peers { permanent_uuid: "904290f5ef46410791dab451856c9239" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38013 } }
I20260806 15:03:59.475361 16563 tablet_bootstrap.cc:492] T 3bf9b68969b5401b801fe462d4b7d591 P 904290f5ef46410791dab451856c9239: No bootstrap required, opened a new log
I20260806 15:03:59.475355 16556 ts_tablet_manager.cc:1428] T 3bf9b68969b5401b801fe462d4b7d591 P cb72f47763f04546a5359cd3fb1dd2ca: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 15:03:59.475417 16563 ts_tablet_manager.cc:1397] T 3bf9b68969b5401b801fe462d4b7d591 P 904290f5ef46410791dab451856c9239: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 15:03:59.475482 16348 heartbeater.cc:503] Master 127.0.236.126:37031 was elected leader, sending a full tablet report...
I20260806 15:03:59.476454 16560 raft_consensus.cc:348] T 3bf9b68969b5401b801fe462d4b7d591 P 173df9b68d5a4e29b631ab8923aa4953 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173df9b68d5a4e29b631ab8923aa4953" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 39145 } } peers { permanent_uuid: "cb72f47763f04546a5359cd3fb1dd2ca" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 35521 } } peers { permanent_uuid: "904290f5ef46410791dab451856c9239" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38013 } }
I20260806 15:03:59.476615 16560 raft_consensus.cc:374] T 3bf9b68969b5401b801fe462d4b7d591 P 173df9b68d5a4e29b631ab8923aa4953 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:59.476657 16560 raft_consensus.cc:729] T 3bf9b68969b5401b801fe462d4b7d591 P 173df9b68d5a4e29b631ab8923aa4953 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 173df9b68d5a4e29b631ab8923aa4953, State: Initialized, Role: FOLLOWER
I20260806 15:03:59.476776 16560 consensus_queue.cc:260] T 3bf9b68969b5401b801fe462d4b7d591 P 173df9b68d5a4e29b631ab8923aa4953 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173df9b68d5a4e29b631ab8923aa4953" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 39145 } } peers { permanent_uuid: "cb72f47763f04546a5359cd3fb1dd2ca" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 35521 } } peers { permanent_uuid: "904290f5ef46410791dab451856c9239" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38013 } }
I20260806 15:03:59.477023 16560 ts_tablet_manager.cc:1428] T 3bf9b68969b5401b801fe462d4b7d591 P 173df9b68d5a4e29b631ab8923aa4953: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 15:03:59.477054 16537 heartbeater.cc:503] Master 127.0.236.126:37031 was elected leader, sending a full tablet report...
I20260806 15:03:59.477174 16563 raft_consensus.cc:348] T 3bf9b68969b5401b801fe462d4b7d591 P 904290f5ef46410791dab451856c9239 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173df9b68d5a4e29b631ab8923aa4953" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 39145 } } peers { permanent_uuid: "cb72f47763f04546a5359cd3fb1dd2ca" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 35521 } } peers { permanent_uuid: "904290f5ef46410791dab451856c9239" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38013 } }
I20260806 15:03:59.477336 16563 raft_consensus.cc:374] T 3bf9b68969b5401b801fe462d4b7d591 P 904290f5ef46410791dab451856c9239 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:03:59.477370 16563 raft_consensus.cc:729] T 3bf9b68969b5401b801fe462d4b7d591 P 904290f5ef46410791dab451856c9239 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 904290f5ef46410791dab451856c9239, State: Initialized, Role: FOLLOWER
I20260806 15:03:59.477471 16563 consensus_queue.cc:260] T 3bf9b68969b5401b801fe462d4b7d591 P 904290f5ef46410791dab451856c9239 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173df9b68d5a4e29b631ab8923aa4953" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 39145 } } peers { permanent_uuid: "cb72f47763f04546a5359cd3fb1dd2ca" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 35521 } } peers { permanent_uuid: "904290f5ef46410791dab451856c9239" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38013 } }
I20260806 15:03:59.477721 16563 ts_tablet_manager.cc:1428] T 3bf9b68969b5401b801fe462d4b7d591 P 904290f5ef46410791dab451856c9239: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 15:03:59.477735 16161 heartbeater.cc:503] Master 127.0.236.126:37031 was elected leader, sending a full tablet report...
W20260806 15:03:59.550076 16349 tablet.cc:2369] T 3bf9b68969b5401b801fe462d4b7d591 P cb72f47763f04546a5359cd3fb1dd2ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:03:59.565915 16568 raft_consensus.cc:482] T 3bf9b68969b5401b801fe462d4b7d591 P 904290f5ef46410791dab451856c9239 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:03:59.566054 16568 raft_consensus.cc:504] T 3bf9b68969b5401b801fe462d4b7d591 P 904290f5ef46410791dab451856c9239 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173df9b68d5a4e29b631ab8923aa4953" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 39145 } } peers { permanent_uuid: "cb72f47763f04546a5359cd3fb1dd2ca" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 35521 } } peers { permanent_uuid: "904290f5ef46410791dab451856c9239" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38013 } }
I20260806 15:03:59.566437 16568 leader_election.cc:288] T 3bf9b68969b5401b801fe462d4b7d591 P 904290f5ef46410791dab451856c9239 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 173df9b68d5a4e29b631ab8923aa4953 (127.0.236.67:39145), cb72f47763f04546a5359cd3fb1dd2ca (127.0.236.66:35521)
I20260806 15:03:59.569214 16552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:59.566737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:43397 (local address 127.0.236.66:35521)
0806 15:03:59.566791 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:59.566793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:59.566801 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:03:59.566829 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:59.566830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:59.566840 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:59.566901 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:59.567090 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.567580 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.568530 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.568894 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.568957 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:59.568967 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:59.569109 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 15:03:59.569147 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 15:03:59.569231 16550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:59.566714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:47425 (local address 127.0.236.67:39145)
0806 15:03:59.566757 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:59.566759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:59.566897 (+ 138us) server_negotiation.cc:407] Connection header received
0806 15:03:59.566967 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:59.566968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:59.566978 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:59.567035 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:03:59.567172 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.567622 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.568530 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.568908 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.568983 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:59.568992 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:59.569123 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 15:03:59.569162 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 15:03:59.569231 16163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:59.566632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:35521 (local address 127.0.236.65:43397)
0806 15:03:59.566674 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:59.566685 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:59.566733 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:59.566938 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:59.566940 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:59.566941 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:59.567021 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:59.567023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.567653 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.567654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:59.568438 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:59.568441 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.568966 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.568967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:59.569045 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:59.569048 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:03:59.569088 (+ 40us) client_negotiation.cc:240] Negotiation successful
0806 15:03:59.569127 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":20}
I20260806 15:03:59.569244 16570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:59.566650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:39145 (local address 127.0.236.65:47425)
0806 15:03:59.566826 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:59.566835 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:03:59.566892 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:03:59.567043 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:03:59.567043 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:03:59.567044 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:03:59.567113 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:59.567114 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.567702 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.567702 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:59.568445 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:03:59.568448 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.568996 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.568997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:03:59.569055 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:59.569059 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:03:59.569109 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 15:03:59.569147 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":1,"thread_start_us":113,"threads_started":1}
I20260806 15:03:59.569585 16285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bf9b68969b5401b801fe462d4b7d591" candidate_uuid: "904290f5ef46410791dab451856c9239" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb72f47763f04546a5359cd3fb1dd2ca" is_pre_election: true
I20260806 15:03:59.569607 16473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bf9b68969b5401b801fe462d4b7d591" candidate_uuid: "904290f5ef46410791dab451856c9239" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173df9b68d5a4e29b631ab8923aa4953" is_pre_election: true
I20260806 15:03:59.569725 16285 raft_consensus.cc:2454] T 3bf9b68969b5401b801fe462d4b7d591 P cb72f47763f04546a5359cd3fb1dd2ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 904290f5ef46410791dab451856c9239 in term 0.
I20260806 15:03:59.569746 16473 raft_consensus.cc:2454] T 3bf9b68969b5401b801fe462d4b7d591 P 173df9b68d5a4e29b631ab8923aa4953 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 904290f5ef46410791dab451856c9239 in term 0.
I20260806 15:03:59.569972 16001 leader_election.cc:302] T 3bf9b68969b5401b801fe462d4b7d591 P 904290f5ef46410791dab451856c9239 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 904290f5ef46410791dab451856c9239, cb72f47763f04546a5359cd3fb1dd2ca; no voters:
I20260806 15:03:59.570138 16568 raft_consensus.cc:2781] T 3bf9b68969b5401b801fe462d4b7d591 P 904290f5ef46410791dab451856c9239 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:03:59.570178 16568 raft_consensus.cc:482] T 3bf9b68969b5401b801fe462d4b7d591 P 904290f5ef46410791dab451856c9239 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:03:59.570199 16568 raft_consensus.cc:3037] T 3bf9b68969b5401b801fe462d4b7d591 P 904290f5ef46410791dab451856c9239 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:59.570859 16568 raft_consensus.cc:504] T 3bf9b68969b5401b801fe462d4b7d591 P 904290f5ef46410791dab451856c9239 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173df9b68d5a4e29b631ab8923aa4953" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 39145 } } peers { permanent_uuid: "cb72f47763f04546a5359cd3fb1dd2ca" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 35521 } } peers { permanent_uuid: "904290f5ef46410791dab451856c9239" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38013 } }
I20260806 15:03:59.571048 16568 leader_election.cc:288] T 3bf9b68969b5401b801fe462d4b7d591 P 904290f5ef46410791dab451856c9239 [CANDIDATE]: Term 1 election: Requested vote from peers 173df9b68d5a4e29b631ab8923aa4953 (127.0.236.67:39145), cb72f47763f04546a5359cd3fb1dd2ca (127.0.236.66:35521)
I20260806 15:03:59.571138 16473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bf9b68969b5401b801fe462d4b7d591" candidate_uuid: "904290f5ef46410791dab451856c9239" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173df9b68d5a4e29b631ab8923aa4953"
I20260806 15:03:59.571142 16285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bf9b68969b5401b801fe462d4b7d591" candidate_uuid: "904290f5ef46410791dab451856c9239" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb72f47763f04546a5359cd3fb1dd2ca"
I20260806 15:03:59.571213 16285 raft_consensus.cc:3037] T 3bf9b68969b5401b801fe462d4b7d591 P cb72f47763f04546a5359cd3fb1dd2ca [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:59.571210 16473 raft_consensus.cc:3037] T 3bf9b68969b5401b801fe462d4b7d591 P 173df9b68d5a4e29b631ab8923aa4953 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:03:59.571869 16473 raft_consensus.cc:2454] T 3bf9b68969b5401b801fe462d4b7d591 P 173df9b68d5a4e29b631ab8923aa4953 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 904290f5ef46410791dab451856c9239 in term 1.
I20260806 15:03:59.571954 16285 raft_consensus.cc:2454] T 3bf9b68969b5401b801fe462d4b7d591 P cb72f47763f04546a5359cd3fb1dd2ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 904290f5ef46410791dab451856c9239 in term 1.
I20260806 15:03:59.572073 16002 leader_election.cc:302] T 3bf9b68969b5401b801fe462d4b7d591 P 904290f5ef46410791dab451856c9239 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 173df9b68d5a4e29b631ab8923aa4953, 904290f5ef46410791dab451856c9239; no voters:
I20260806 15:03:59.572181 16568 raft_consensus.cc:2781] T 3bf9b68969b5401b801fe462d4b7d591 P 904290f5ef46410791dab451856c9239 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:03:59.572330 16568 raft_consensus.cc:686] T 3bf9b68969b5401b801fe462d4b7d591 P 904290f5ef46410791dab451856c9239 [term 1 LEADER]: Becoming Leader. State: Replica: 904290f5ef46410791dab451856c9239, State: Running, Role: LEADER
I20260806 15:03:59.572427 16568 consensus_queue.cc:237] T 3bf9b68969b5401b801fe462d4b7d591 P 904290f5ef46410791dab451856c9239 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173df9b68d5a4e29b631ab8923aa4953" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 39145 } } peers { permanent_uuid: "cb72f47763f04546a5359cd3fb1dd2ca" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 35521 } } peers { permanent_uuid: "904290f5ef46410791dab451856c9239" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38013 } }
I20260806 15:03:59.573196 15900 catalog_manager.cc:5495] T 3bf9b68969b5401b801fe462d4b7d591 P 904290f5ef46410791dab451856c9239 reported cstate change: term changed from 0 to 1, leader changed from <none> to 904290f5ef46410791dab451856c9239 (127.0.236.65). New cstate: current_term: 1 leader_uuid: "904290f5ef46410791dab451856c9239" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173df9b68d5a4e29b631ab8923aa4953" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 39145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb72f47763f04546a5359cd3fb1dd2ca" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 35521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "904290f5ef46410791dab451856c9239" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 38013 } health_report { overall_health: HEALTHY } } }
I20260806 15:03:59.581837 16554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:59.579813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54502 (local address 127.0.236.65:38013)
0806 15:03:59.579856 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 15:03:59.579857 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:03:59.579866 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:03:59.579926 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:03:59.579926 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:03:59.579940 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:03:59.579997 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:03:59.580097 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.580533 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.581198 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:03:59.581364 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:03:59.581402 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:03:59.581598 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:03:59.581681 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:03:59.581732 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 15:03:59.581760 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 15:03:59.581775 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 15:03:59.583381 16285 raft_consensus.cc:1261] T 3bf9b68969b5401b801fe462d4b7d591 P cb72f47763f04546a5359cd3fb1dd2ca [term 1 FOLLOWER]: Refusing update from remote peer 904290f5ef46410791dab451856c9239: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:59.583405 16473 raft_consensus.cc:1261] T 3bf9b68969b5401b801fe462d4b7d591 P 173df9b68d5a4e29b631ab8923aa4953 [term 1 FOLLOWER]: Refusing update from remote peer 904290f5ef46410791dab451856c9239: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:03:59.583666 16568 consensus_queue.cc:1035] T 3bf9b68969b5401b801fe462d4b7d591 P 904290f5ef46410791dab451856c9239 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb72f47763f04546a5359cd3fb1dd2ca" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 35521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:59.583794 16572 consensus_queue.cc:1035] T 3bf9b68969b5401b801fe462d4b7d591 P 904290f5ef46410791dab451856c9239 [LEADER]: Connected to new peer: Peer: permanent_uuid: "173df9b68d5a4e29b631ab8923aa4953" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 39145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:03:59.585747 16576 mvcc.cc:205] Tried to move back new op lower bound from 7315573307731251200 to 7315573307689218048. Current Snapshot: MvccSnapshot[applied={T|T < 7315573307731251200}]
I20260806 15:03:59.586289 16579 mvcc.cc:205] Tried to move back new op lower bound from 7315573307731251200 to 7315573307689218048. Current Snapshot: MvccSnapshot[applied={T|T < 7315573307731251200}]
I20260806 15:03:59.586513 16577 mvcc.cc:205] Tried to move back new op lower bound from 7315573307731251200 to 7315573307689218048. Current Snapshot: MvccSnapshot[applied={T|T < 7315573307731251200}]
I20260806 15:03:59.588295 15900 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:59.588373 15900 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:03:59.589007 15900 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 251312685abd4500820af23bee9689cb: Sending DeleteTablet for 3 replicas of tablet 3bf9b68969b5401b801fe462d4b7d591
I20260806 15:03:59.589329 16065 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bf9b68969b5401b801fe462d4b7d591 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:59 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:54492
I20260806 15:03:59.589354 16447 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bf9b68969b5401b801fe462d4b7d591 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:59 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:51110
I20260806 15:03:59.589366 16258 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bf9b68969b5401b801fe462d4b7d591 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:03:59 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:44084
I20260806 15:03:59.589651 16590 tablet_replica.cc:330] T 3bf9b68969b5401b801fe462d4b7d591 P 173df9b68d5a4e29b631ab8923aa4953: stopping tablet replica
I20260806 15:03:59.589790 16590 raft_consensus.cc:2229] T 3bf9b68969b5401b801fe462d4b7d591 P 173df9b68d5a4e29b631ab8923aa4953 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:59.589804 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 15982
I20260806 15:03:59.590301 16591 tablet_replica.cc:330] T 3bf9b68969b5401b801fe462d4b7d591 P cb72f47763f04546a5359cd3fb1dd2ca: stopping tablet replica
I20260806 15:03:59.590402 16591 raft_consensus.cc:2229] T 3bf9b68969b5401b801fe462d4b7d591 P cb72f47763f04546a5359cd3fb1dd2ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:03:59.590413 16590 raft_consensus.cc:2258] T 3bf9b68969b5401b801fe462d4b7d591 P 173df9b68d5a4e29b631ab8923aa4953 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:59.590498 16591 raft_consensus.cc:2258] T 3bf9b68969b5401b801fe462d4b7d591 P cb72f47763f04546a5359cd3fb1dd2ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:03:59.600579 16591 ts_tablet_manager.cc:1905] T 3bf9b68969b5401b801fe462d4b7d591 P cb72f47763f04546a5359cd3fb1dd2ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:59.600780 16590 ts_tablet_manager.cc:1905] T 3bf9b68969b5401b801fe462d4b7d591 P 173df9b68d5a4e29b631ab8923aa4953: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:03:59.602375 16591 ts_tablet_manager.cc:1918] T 3bf9b68969b5401b801fe462d4b7d591 P cb72f47763f04546a5359cd3fb1dd2ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:03:59.602384 16590 ts_tablet_manager.cc:1918] T 3bf9b68969b5401b801fe462d4b7d591 P 173df9b68d5a4e29b631ab8923aa4953: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:03:59.602455 16591 log.cc:1196] T 3bf9b68969b5401b801fe462d4b7d591 P cb72f47763f04546a5359cd3fb1dd2ca: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-1/wal/wals/3bf9b68969b5401b801fe462d4b7d591
I20260806 15:03:59.602456 16590 log.cc:1196] T 3bf9b68969b5401b801fe462d4b7d591 P 173df9b68d5a4e29b631ab8923aa4953: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/3bf9b68969b5401b801fe462d4b7d591
I20260806 15:03:59.602728 16590 ts_tablet_manager.cc:1939] T 3bf9b68969b5401b801fe462d4b7d591 P 173df9b68d5a4e29b631ab8923aa4953: Deleting consensus metadata
I20260806 15:03:59.602728 16591 ts_tablet_manager.cc:1939] T 3bf9b68969b5401b801fe462d4b7d591 P cb72f47763f04546a5359cd3fb1dd2ca: Deleting consensus metadata
I20260806 15:03:59.603118 15876 catalog_manager.cc:4841] TS 173df9b68d5a4e29b631ab8923aa4953 (127.0.236.67:39145): tablet 3bf9b68969b5401b801fe462d4b7d591 (table test-table [id=c642edff5e4c484db8abf2ca0ef34d0f]) successfully deleted
I20260806 15:03:59.603157 15875 catalog_manager.cc:4841] TS cb72f47763f04546a5359cd3fb1dd2ca (127.0.236.66:35521): tablet 3bf9b68969b5401b801fe462d4b7d591 (table test-table [id=c642edff5e4c484db8abf2ca0ef34d0f]) successfully deleted
W20260806 15:03:59.604921 15875 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.236.65:38013 (error 108)
I20260806 15:03:59.605361 16553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:03:59.605241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:38013 (local address 127.0.0.1:54516)
0806 15:03:59.605298 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 15:03:59.605332 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.65:38013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 15:03:59.605443 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 16166
W20260806 15:03:59.605544 15875 catalog_manager.cc:4584] TS 904290f5ef46410791dab451856c9239 (127.0.236.65:38013): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3bf9b68969b5401b801fe462d4b7d591: Network error: Client connection negotiation failed: client connection to 127.0.236.65:38013: connect: Connection refused (error 111)
I20260806 15:03:59.613610 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 16353
I20260806 15:03:59.623232 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 15849
2026-08-06T15:03:59Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2014 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:03:59 dist-test-slave-tc2s krb5kdc[16599](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:03:59 dist-test-slave-tc2s krb5kdc[16599](info): set up 2 sockets
Aug 06 15:03:59 dist-test-slave-tc2s krb5kdc[16599](info): commencing operation
krb5kdc: starting...
W20260806 15:04:00.730054 945 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:04:00 dist-test-slave-tc2s krb5kdc[16599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:04:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:04:00Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.0.236.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.0.236.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.0.236.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/rangeradmin_127.0.236.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.0.236.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/rangeradmin_127.0.236.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.0.236.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.0.236.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.0.236.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/rangerlookup_127.0.236.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.0.236.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/rangerlookup_127.0.236.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.236.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.0.236.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.236.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/HTTP_127.0.236.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.236.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/HTTP_127.0.236.84@KRBTEST.COM.keytab.
I20260806 15:04:00.877749 945 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskFIWuwX/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/postgres -l logfile start
2026-08-06 15:04:03.001 UTC [16636] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 15:04:03.001 UTC [16636] LOG: listening on IPv4 address "127.0.236.84", port 39793
2026-08-06 15:04:03.004 UTC [16636] LOG: listening on Unix socket "/tmp/.s.PGSQL.39793"
2026-08-06 15:04:03.023 UTC [16639] LOG: database system was shut down at 2026-08-06 15:04:01 UTC
2026-08-06 15:04:03.026 UTC [16636] LOG: database system is ready to accept connections
I20260806 15:04:04.054388 945 mini_postgres.cc:96] Postgres bound to 39793
2026-08-06 15:04:04.061 UTC [16648] FATAL: database "slave" does not exist
127.0.236.84:39793 - accepting connections
I20260806 15:04:04.062559 945 mini_ranger.cc:161] Starting Ranger...
I20260806 15:04:04.077158 945 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 15:04:04.456583 945 mini_ranger.cc:87] Created ranger Postgres database
I20260806 15:04:04.456650 945 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ranger-admin
2026-08-06 15:04:04,783 [I] DB FLAVOR :POSTGRES
2026-08-06 15:04:04,783 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 15:04:04,783 [I] Checking connection..
2026-08-06 15:04:04,784 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:39793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 15:04:05,039 [I] Checking connection passed.
2026-08-06 15:04:05,040 [I] --------- Verifying version history table ---------
2026-08-06 15:04:05,040 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:39793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 15:04:05,289 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 15:04:05,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:39793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 15:04:05,517 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 15:04:05,518 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 15:04:05,518 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:39793/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 15:04:05.760 UTC [16746] WARNING: there is no transaction in progress
2026-08-06 15:04:05,768 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 15:04:05,768 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:39793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 15:04:06,011 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 15:04:06,011 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 15:04:06,012 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:39793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 15:04:06,261 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:39793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 15:04:06,503 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:39793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-tc2s.c.gcp-upstream.internal','N') ;"
2026-08-06 15:04:06,741 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:39793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 15:04:06,982 [I] Table x_portal_user does not exist in database ranger
2026-08-06 15:04:06,983 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:39793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 15:04:07,224 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 15:04:07,225 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:39793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 15:04:07,458 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:39793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 15:04:07,707 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 15:04:07,707 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:39793/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 15:04:08.573 UTC [16898] WARNING: there is no transaction in progress
2026-08-06 15:04:08.588 UTC [16898] WARNING: there is no transaction in progress
2026-08-06 15:04:08.603 UTC [16898] WARNING: there is no transaction in progress
2026-08-06 15:04:08.616 UTC [16898] WARNING: there is no transaction in progress
2026-08-06 15:04:08.631 UTC [16898] WARNING: there is no transaction in progress
2026-08-06 15:04:08.647 UTC [16898] WARNING: there is no transaction in progress
2026-08-06 15:04:08.695 UTC [16898] WARNING: there is no transaction in progress
2026-08-06 15:04:08.701 UTC [16898] WARNING: there is no transaction in progress
2026-08-06 15:04:08.712 UTC [16898] WARNING: there is no transaction in progress
2026-08-06 15:04:08.721 UTC [16898] WARNING: there is no transaction in progress
2026-08-06 15:04:08.730 UTC [16898] WARNING: there is no transaction in progress
2026-08-06 15:04:08.742 UTC [16898] WARNING: there is no transaction in progress
2026-08-06 15:04:08.750 UTC [16898] WARNING: there is no transaction in progress
2026-08-06 15:04:08.755 UTC [16898] WARNING: there is no transaction in progress
2026-08-06 15:04:08.761 UTC [16898] WARNING: there is no transaction in progress
2026-08-06 15:04:08.767 UTC [16898] WARNING: there is no transaction in progress
2026-08-06 15:04:09,285 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 15:04:09,285 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:39793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 15:04:09,527 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 15:04:09,528 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:39793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 15:04:09,782 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 15:04:09,783 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:39793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 15:04:10,009 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 15:04:10,009 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:39793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 15:04:10,232 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:39793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 15:04:10,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:39793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-tc2s.c.gcp-upstream.internal';"
2026-08-06 15:04:10,699 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 15:04:10,699 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:39793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 15:04:10,945 [I] DB_PATCHES have already been applied
I20260806 15:04:10.950532 945 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ranger-admin:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskFIWuwX/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 15:04:10.950590 945 mini_ranger.cc:193] Using host: 127.0.236.84
I20260806 15:04:10.954075 945 mini_ranger.cc:227] Ranger admin URL: http://127.0.236.84:47483
Aug 06, 2026 3:04:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 3:04:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 3:04:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 3:04:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 3:04:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 3:04:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 3:04:11 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 15:04:12.141014 945 mini_ranger.cc:160] Time spent starting Ranger: real 8.078s user 0.000s sys 0.010s
Aug 06, 2026 3:04:31 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 3:04:31 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 3:04:31 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 3:04:31 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 3:04:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 3:04:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 3:04:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 3:04:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 3:04:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 3:04:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 3:04:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 3:04:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 3:04:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 3:04:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 3:04:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 3:04:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 3:04:31 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 3:04:32 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 15:04:32.909596 945 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.0.236.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.0.236.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.0.236.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/rangerkms_127.0.236.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.0.236.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/rangerkms_127.0.236.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.236.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.0.236.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.236.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/HTTP_127.0.236.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.236.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/HTTP_127.0.236.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 15:04:32 dist-test-slave-tc2s krb5kdc[16599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028672, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 15:04:32.987882 945 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 15:04:32.996438 945 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 15:04:33.637005 945 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 15:04:33.637089 945 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ranger-kms
I20260806 15:04:33.637106 945 mini_ranger_kms.cc:227] Using postgres at 127.0.236.84:39793
2026-08-06 15:04:33,887 [I] DB FLAVOR :POSTGRES
2026-08-06 15:04:33,887 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 15:04:33,887 [I] Checking connection
2026-08-06 15:04:33,887 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:39793/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 15:04:34,141 [I] connection success
2026-08-06 15:04:34,141 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 15:04:34,141 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 15:04:34,141 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:39793/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 15:04:34,387 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 15:04:34,387 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 15:04:34,387 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 15:04:34,388 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:39793/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 15:04:34,628 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 15:04:35.081107 945 mini_ranger_kms.cc:310] Created kms service
I20260806 15:04:35.263674 945 mini_ranger_kms.cc:326] Created kudu user
I20260806 15:04:35.305408 945 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 15:04:35 dist-test-slave-tc2s krb5kdc[16599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028672, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.0.236.84@KRBTEST.COM
I20260806 15:04:35.685633 945 mini_ranger_kms.cc:384] Added ranger policy
I20260806 15:04:35.685750 945 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ranger-kms:/tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 15:04:35.685771 945 mini_ranger_kms.cc:240] Using host: 127.0.236.84
I20260806 15:04:35.689632 945 mini_ranger_kms.cc:276] Ranger KMS PID: 17153
I20260806 15:04:35.689750 945 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.0.236.84:57441
Aug 06, 2026 3:04:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 3:04:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 3:04:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 3:04:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 3:04:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 3:04:37 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 15:04:37 dist-test-slave-tc2s krb5kdc[16599](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786028677, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.236.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:04:38.072294 945 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.084s user 0.009s sys 0.222s
I20260806 15:04:38.072409 945 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 15:04:38.072490 945 mini_ranger_kms.cc:401] 127.0.236.84:57441/kms/v1/keys
Aug 06 15:04:38 dist-test-slave-tc2s krb5kdc[16599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028677, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.236.84@KRBTEST.COM for HTTP/127.0.236.84@KRBTEST.COM
Aug 06 15:04:38 dist-test-slave-tc2s krb5kdc[16599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028677, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.236.84@KRBTEST.COM for HTTP/127.0.236.84@KRBTEST.COM
Aug 06 15:04:40 dist-test-slave-tc2s krb5kdc[16599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:04:40.710086 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:37465
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:37465
--ranger_config_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.236.84:57441/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:04:40.788640 17226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:04:40.788828 17226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:04:40.788858 17226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:04:40.789988 17226 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:04:40.790026 17226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:04:40.790046 17226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:04:40.790062 17226 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:04:40.790088 17226 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:04:40.791667 17226 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/master-0/wal
--ranger_kms_url=127.0.236.84:57441/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:37465
--ranger_config_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:37465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:04:40.791877 17226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:04:40.792070 17226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:04:40.796852 17237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:04:40.796815 17235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:04:40.796949 17226 server_base.cc:1034] running on GCE node
W20260806 15:04:40.796804 17232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:04:40.797358 17226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:04:40.797627 17226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:04:40.798799 17226 hybrid_clock.cc:648] HybridClock initialized: now 1786028680798785 us; error 31 us; skew 500 ppm
Aug 06 15:04:40 dist-test-slave-tc2s krb5kdc[16599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028680, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:04:40.801415 17226 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:04:40.801988 17226 webserver.cc:460] Webserver started at http://127.0.236.126:32781/ using document root <none> and password file <none>
I20260806 15:04:40.802289 17226 fs_manager.cc:359] Metadata directory not provided
I20260806 15:04:40.802348 17226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:04:40.802488 17226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 15:04:40 dist-test-slave-tc2s krb5kdc[16599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028680, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for HTTP/127.0.236.84@KRBTEST.COM
I20260806 15:04:40.851874 17226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "ed30a8de21fb480da1d6dc2e6ec082a9"
format_stamp: "Formatted at 2026-08-06 15:04:40 on dist-test-slave-tc2s"
server_key: "b1215697d868df5b708bb9e077856362"
server_key_iv: "644e0bc186e291a51adc7bdfad9ff092"
server_key_version: "kuduclusterkey@0"
I20260806 15:04:40.852316 17226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "ed30a8de21fb480da1d6dc2e6ec082a9"
format_stamp: "Formatted at 2026-08-06 15:04:40 on dist-test-slave-tc2s"
server_key: "b1215697d868df5b708bb9e077856362"
server_key_iv: "644e0bc186e291a51adc7bdfad9ff092"
server_key_version: "kuduclusterkey@0"
I20260806 15:04:40.854210 17226 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20260806 15:04:40.864961 17245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:04:40.865152 17226 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 15:04:40.865242 17226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "ed30a8de21fb480da1d6dc2e6ec082a9"
format_stamp: "Formatted at 2026-08-06 15:04:40 on dist-test-slave-tc2s"
server_key: "b1215697d868df5b708bb9e077856362"
server_key_iv: "644e0bc186e291a51adc7bdfad9ff092"
server_key_version: "kuduclusterkey@0"
I20260806 15:04:40.865321 17226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:04:40.874552 17226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:04:40.876277 17226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:04:40.876406 17226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:04:40.880287 17226 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:37465
I20260806 15:04:40.880353 17322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:37465 every 8 connection(s)
I20260806 15:04:40.880669 17226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:04:40.881421 17323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:04:40.884513 17323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed30a8de21fb480da1d6dc2e6ec082a9: Bootstrap starting.
I20260806 15:04:40.885015 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 17226
I20260806 15:04:40.885109 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:04:40.885205 17323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed30a8de21fb480da1d6dc2e6ec082a9: Neither blocks nor log segments found. Creating new log.
I20260806 15:04:40.885286 945 external_mini_cluster.cc:1428] Setting key 9b0b7cbdf242f5715aa193ca5daf4948
I20260806 15:04:40.885535 17323 log.cc:824] T 00000000000000000000000000000000 P ed30a8de21fb480da1d6dc2e6ec082a9: Log is configured to *not* fsync() on all Append() calls
I20260806 15:04:40.886363 17323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed30a8de21fb480da1d6dc2e6ec082a9: No bootstrap required, opened a new log
I20260806 15:04:40.887969 17323 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed30a8de21fb480da1d6dc2e6ec082a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed30a8de21fb480da1d6dc2e6ec082a9" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 37465 } }
I20260806 15:04:40.888131 17323 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed30a8de21fb480da1d6dc2e6ec082a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:04:40.888163 17323 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed30a8de21fb480da1d6dc2e6ec082a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed30a8de21fb480da1d6dc2e6ec082a9, State: Initialized, Role: FOLLOWER
I20260806 15:04:40.888278 17323 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed30a8de21fb480da1d6dc2e6ec082a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed30a8de21fb480da1d6dc2e6ec082a9" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 37465 } }
I20260806 15:04:40.888350 17323 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed30a8de21fb480da1d6dc2e6ec082a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:04:40.888376 17323 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed30a8de21fb480da1d6dc2e6ec082a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:04:40.888412 17323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed30a8de21fb480da1d6dc2e6ec082a9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:04:40.889032 17323 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed30a8de21fb480da1d6dc2e6ec082a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed30a8de21fb480da1d6dc2e6ec082a9" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 37465 } }
I20260806 15:04:40.889120 17323 leader_election.cc:302] T 00000000000000000000000000000000 P ed30a8de21fb480da1d6dc2e6ec082a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed30a8de21fb480da1d6dc2e6ec082a9; no voters:
I20260806 15:04:40.889262 17323 leader_election.cc:288] T 00000000000000000000000000000000 P ed30a8de21fb480da1d6dc2e6ec082a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:04:40.889326 17329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed30a8de21fb480da1d6dc2e6ec082a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:04:40.889483 17329 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed30a8de21fb480da1d6dc2e6ec082a9 [term 1 LEADER]: Becoming Leader. State: Replica: ed30a8de21fb480da1d6dc2e6ec082a9, State: Running, Role: LEADER
I20260806 15:04:40.889500 17323 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed30a8de21fb480da1d6dc2e6ec082a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:04:40.889564 17329 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed30a8de21fb480da1d6dc2e6ec082a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed30a8de21fb480da1d6dc2e6ec082a9" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 37465 } }
I20260806 15:04:40.889916 17329 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed30a8de21fb480da1d6dc2e6ec082a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed30a8de21fb480da1d6dc2e6ec082a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed30a8de21fb480da1d6dc2e6ec082a9" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 37465 } } }
I20260806 15:04:40.890017 17329 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed30a8de21fb480da1d6dc2e6ec082a9 [sys.catalog]: This master's current role is: LEADER
I20260806 15:04:40.890158 17329 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed30a8de21fb480da1d6dc2e6ec082a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed30a8de21fb480da1d6dc2e6ec082a9. Latest consensus state: current_term: 1 leader_uuid: "ed30a8de21fb480da1d6dc2e6ec082a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed30a8de21fb480da1d6dc2e6ec082a9" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 37465 } } }
I20260806 15:04:40.890277 17329 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed30a8de21fb480da1d6dc2e6ec082a9 [sys.catalog]: This master's current role is: LEADER
I20260806 15:04:40.890432 17338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:04:40.891021 17338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:04:40.891572 17323 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 06 15:04:40 dist-test-slave-tc2s krb5kdc[16599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:04:40.899806 17327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:04:40.885736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51454 (local address 127.0.236.126:37465)
0806 15:04:40.885932 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 15:04:40.885941 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:04:40.885988 (+ 47us) server_negotiation.cc:407] Connection header received
0806 15:04:40.886531 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:04:40.886540 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:04:40.886598 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:04:40.886743 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:04:40.887481 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:04:40.889182 (+ 1701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:04:40.890297 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:04:40.890935 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:04:40.891037 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:04:40.894287 (+ 3250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:04:40.894314 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:04:40.894315 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:04:40.894395 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:04:40.897311 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:04:40.897703 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:04:40.897706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:04:40.897707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:04:40.897811 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:04:40.898065 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:04:40.898066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:04:40.898066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:04:40.898515 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:04:40.898703 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 15:04:40.899231 (+ 528us) server_negotiation.cc:299] Negotiation successful
0806 15:04:40.899523 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":77,"threads_started":1}
I20260806 15:04:40.906621 17338 catalog_manager.cc:1347] Generated new cluster ID: 57bfe131eeac4da59c776e6259d89f1a
I20260806 15:04:40.906677 17338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:04:40.928223 17338 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:04:40.929036 17338 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:04:40.937813 17338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed30a8de21fb480da1d6dc2e6ec082a9: Generated new TSK 0
I20260806 15:04:40.938090 17338 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.236.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
I20260806 15:04:41.618770 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.236.84:57441/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:0
--local_ip_for_outbound_sockets=127.0.236.65
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:37465
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:04:41.710336 17386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:04:41.710515 17386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:04:41.710544 17386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:04:41.711634 17386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:04:41.711673 17386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:04:41.711706 17386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:04:41.713117 17386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.0.236.84:57441/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:37465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:04:41.713325 17386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:04:41.713541 17386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:04:41.719182 17392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:04:41.719190 17394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:04:41.719405 17386 server_base.cc:1034] running on GCE node
W20260806 15:04:41.719496 17396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:04:41.719677 17386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:04:41.719907 17386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:04:41.721047 17386 hybrid_clock.cc:648] HybridClock initialized: now 1786028681721035 us; error 31 us; skew 500 ppm
Aug 06 15:04:41 dist-test-slave-tc2s krb5kdc[16599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028681, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:04:41.723646 17386 init.cc:377] Logged in from keytab as kudu/127.0.236.65@KRBTEST.COM (short username kudu)
I20260806 15:04:41.724036 17386 webserver.cc:460] Webserver started at http://127.0.236.65:35033/ using document root <none> and password file <none>
I20260806 15:04:41.724254 17386 fs_manager.cc:359] Metadata directory not provided
I20260806 15:04:41.724298 17386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:04:41.724404 17386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 15:04:41 dist-test-slave-tc2s krb5kdc[16599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028681, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for HTTP/127.0.236.84@KRBTEST.COM
I20260806 15:04:41.737561 17386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-0/data/instance:
uuid: "1111f3c7318c4883a2cbb733ca2b4639"
format_stamp: "Formatted at 2026-08-06 15:04:41 on dist-test-slave-tc2s"
server_key: "42959e338ba72984e4c912aca0328b53"
server_key_iv: "4cffaf968aa2651de750eea1a5c48d29"
server_key_version: "kuduclusterkey@0"
I20260806 15:04:41.739183 17386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-0/wal/instance:
uuid: "1111f3c7318c4883a2cbb733ca2b4639"
format_stamp: "Formatted at 2026-08-06 15:04:41 on dist-test-slave-tc2s"
server_key: "42959e338ba72984e4c912aca0328b53"
server_key_iv: "4cffaf968aa2651de750eea1a5c48d29"
server_key_version: "kuduclusterkey@0"
I20260806 15:04:41.740651 17386 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:04:41.760408 17407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:04:41.760656 17386 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260806 15:04:41.760771 17386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "1111f3c7318c4883a2cbb733ca2b4639"
format_stamp: "Formatted at 2026-08-06 15:04:41 on dist-test-slave-tc2s"
server_key: "42959e338ba72984e4c912aca0328b53"
server_key_iv: "4cffaf968aa2651de750eea1a5c48d29"
server_key_version: "kuduclusterkey@0"
I20260806 15:04:41.760861 17386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:04:41.788574 17386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:04:41.790668 17386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:04:41.790841 17386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:04:41.791102 17386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:04:41.791476 17386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:04:41.791523 17386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:04:41.791561 17386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:04:41.791585 17386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:04:41.799436 17386 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:35319
I20260806 15:04:41.799499 17579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:35319 every 8 connection(s)
I20260806 15:04:41.799854 17386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:04:41.804410 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 17386
I20260806 15:04:41.804522 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-0/wal/instance
I20260806 15:04:41.804702 945 external_mini_cluster.cc:1428] Setting key 68bfb419a18d03aecee338868a18a179
Aug 06 15:04:41 dist-test-slave-tc2s krb5kdc[16599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028681, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:04:41.813076 17586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:04:41.800833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:56053 (local address 127.0.236.126:37465)
0806 15:04:41.801728 (+ 895us) server_negotiation.cc:206] Beginning negotiation
0806 15:04:41.801730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:04:41.801740 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:04:41.801766 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:04:41.801767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:04:41.801781 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:04:41.801854 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:04:41.802674 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:04:41.803461 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:04:41.807634 (+ 4173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:04:41.807934 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:04:41.808007 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:04:41.810146 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:04:41.810148 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:04:41.810148 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:04:41.810155 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:04:41.811687 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:04:41.812060 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:04:41.812061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:04:41.812061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:04:41.812230 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:04:41.812470 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:04:41.812471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:04:41.812471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:04:41.812577 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:04:41.812630 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 15:04:41.812898 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 15:04:41.812981 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":862,"thread_start_us":78,"threads_started":1}
I20260806 15:04:41.813217 17585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:04:41.800574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:37465 (local address 127.0.236.65:56053)
0806 15:04:41.800746 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 15:04:41.800798 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 15:04:41.801418 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:04:41.802011 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:04:41.802015 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:04:41.802263 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:04:41.802553 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:04:41.802560 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:04:41.803565 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:04:41.803566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:04:41.807538 (+ 3972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:04:41.807542 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:04:41.808030 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:04:41.808031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:04:41.808175 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:04:41.808732 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:04:41.808740 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:04:41.810038 (+ 1298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:04:41.811765 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:04:41.811771 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:04:41.811773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:04:41.811985 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:04:41.812293 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:04:41.812294 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:04:41.812294 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:04:41.812404 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:04:41.812633 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:04:41.812639 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:04:41.812769 (+ 130us) client_negotiation.cc:769] Sending connection context
0806 15:04:41.812855 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 15:04:41.813002 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":84,"threads_started":1}
I20260806 15:04:41.813733 17583 heartbeater.cc:347] Connected to a master server at 127.0.236.126:37465
I20260806 15:04:41.813827 17583 heartbeater.cc:464] Registering TS with master...
I20260806 15:04:41.814035 17583 heartbeater.cc:511] Master 127.0.236.126:37465 requested a full tablet report, sending...
I20260806 15:04:41.814522 17271 ts_manager.cc:194] Registered new tserver with Master: 1111f3c7318c4883a2cbb733ca2b4639 (127.0.236.65:35319)
I20260806 15:04:41.815088 17271 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.65@KRBTEST.COM'} at 127.0.236.65:56053
WARNING: no policy specified for kudu/127.0.236.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
I20260806 15:04:41.836355 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.236.84:57441/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:0
--local_ip_for_outbound_sockets=127.0.236.66
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:37465
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:04:41.946998 17592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:04:41.947190 17592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:04:41.947219 17592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:04:41.948339 17592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:04:41.948379 17592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:04:41.948413 17592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:04:41.949832 17592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.0.236.84:57441/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:37465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:04:41.950068 17592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:04:41.950304 17592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:04:41.955005 17606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:04:41.955005 17602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:04:41.955222 17601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:04:41.955922 17592 server_base.cc:1034] running on GCE node
I20260806 15:04:41.956120 17592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:04:41.956334 17592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:04:41.957479 17592 hybrid_clock.cc:648] HybridClock initialized: now 1786028681957471 us; error 28 us; skew 500 ppm
Aug 06 15:04:41 dist-test-slave-tc2s krb5kdc[16599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028681, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:04:41.959975 17592 init.cc:377] Logged in from keytab as kudu/127.0.236.66@KRBTEST.COM (short username kudu)
I20260806 15:04:41.960354 17592 webserver.cc:460] Webserver started at http://127.0.236.66:42371/ using document root <none> and password file <none>
I20260806 15:04:41.960554 17592 fs_manager.cc:359] Metadata directory not provided
I20260806 15:04:41.960600 17592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:04:41.960711 17592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 15:04:41 dist-test-slave-tc2s krb5kdc[16599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028681, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for HTTP/127.0.236.84@KRBTEST.COM
I20260806 15:04:41.971792 17592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-1/data/instance:
uuid: "cdc631647c3b48de8f1249ee6fd7e1d9"
format_stamp: "Formatted at 2026-08-06 15:04:41 on dist-test-slave-tc2s"
server_key: "e2021dbcb38dbc902b3a76dcdc2f35ab"
server_key_iv: "f0166237b7037149960336a1183cadbe"
server_key_version: "kuduclusterkey@0"
I20260806 15:04:41.972375 17592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-1/wal/instance:
uuid: "cdc631647c3b48de8f1249ee6fd7e1d9"
format_stamp: "Formatted at 2026-08-06 15:04:41 on dist-test-slave-tc2s"
server_key: "e2021dbcb38dbc902b3a76dcdc2f35ab"
server_key_iv: "f0166237b7037149960336a1183cadbe"
server_key_version: "kuduclusterkey@0"
I20260806 15:04:41.974364 17592 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
Aug 06 15:04:41 dist-test-slave-tc2s krb5kdc[16599](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786028681, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:04:41.994015 17615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:04:41.994234 17592 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260806 15:04:41.994328 17592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "cdc631647c3b48de8f1249ee6fd7e1d9"
format_stamp: "Formatted at 2026-08-06 15:04:41 on dist-test-slave-tc2s"
server_key: "e2021dbcb38dbc902b3a76dcdc2f35ab"
server_key_iv: "f0166237b7037149960336a1183cadbe"
server_key_version: "kuduclusterkey@0"
I20260806 15:04:41.994416 17592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:04:42.011220 17592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:04:42.013485 17592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:04:42.013691 17592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:04:42.013988 17592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:04:42.014410 17592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:04:42.014503 17592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:04:42.014576 17592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:04:42.014636 17592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:04:42.020908 17592 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:43245
I20260806 15:04:42.021368 17592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
I20260806 15:04:42.022596 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 17592
I20260806 15:04:42.022693 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-1/wal/instance
I20260806 15:04:42.022907 945 external_mini_cluster.cc:1428] Setting key c828379699a796ba01105cf6f6051f81
I20260806 15:04:42.026914 17777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:43245 every 8 connection(s)
Aug 06 15:04:42 dist-test-slave-tc2s krb5kdc[16599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028681, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:04:42.039343 17586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:04:42.027760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:54905 (local address 127.0.236.126:37465)
0806 15:04:42.027812 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 15:04:42.027815 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:04:42.029422 (+ 1607us) server_negotiation.cc:407] Connection header received
0806 15:04:42.030117 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:04:42.030119 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:04:42.030134 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:04:42.030165 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:04:42.030989 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.031745 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.033230 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.033506 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.033553 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:04:42.036142 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:04:42.036144 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:04:42.036144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:04:42.036151 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:04:42.037754 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:04:42.038261 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:04:42.038262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:04:42.038262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:04:42.038327 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:04:42.038598 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:04:42.038599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:04:42.038599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:04:42.038736 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:04:42.038790 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 15:04:42.039158 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 15:04:42.039250 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 15:04:42.040057 17781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:04:42.027512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:37465 (local address 127.0.236.66:54905)
0806 15:04:42.029290 (+ 1778us) negotiation.cc:107] Waiting for socket to connect
0806 15:04:42.029337 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 15:04:42.029994 (+ 657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:04:42.030347 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:04:42.030352 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:04:42.030599 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:04:42.030879 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:04:42.030885 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.031877 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.031878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:04:42.033130 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:04:42.033132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.033644 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.033645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:04:42.033786 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:04:42.034351 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:04:42.034361 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:04:42.036033 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:04:42.037854 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:04:42.037862 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:04:42.037865 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:04:42.038169 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:04:42.038395 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:04:42.038395 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:04:42.038396 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:04:42.038522 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:04:42.038825 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:04:42.038832 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:04:42.039000 (+ 168us) client_negotiation.cc:769] Sending connection context
0806 15:04:42.039670 (+ 670us) client_negotiation.cc:240] Negotiation successful
0806 15:04:42.039828 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1730,"thread_start_us":72,"threads_started":1}
WARNING: no policy specified for kudu/127.0.236.67@KRBTEST.COM; defaulting to no policy
I20260806 15:04:42.040727 17778 heartbeater.cc:347] Connected to a master server at 127.0.236.126:37465
I20260806 15:04:42.040818 17778 heartbeater.cc:464] Registering TS with master...
I20260806 15:04:42.041007 17778 heartbeater.cc:511] Master 127.0.236.126:37465 requested a full tablet report, sending...
I20260806 15:04:42.041414 17271 ts_manager.cc:194] Registered new tserver with Master: cdc631647c3b48de8f1249ee6fd7e1d9 (127.0.236.66:43245)
I20260806 15:04:42.041914 17271 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.66@KRBTEST.COM'} at 127.0.236.66:54905
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
I20260806 15:04:42.066413 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.236.84:57441/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:0
--local_ip_for_outbound_sockets=127.0.236.67
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:37465
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:04:42.188233 17783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:04:42.188570 17783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:04:42.188657 17783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:04:42.190400 17783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:04:42.190611 17783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:04:42.190702 17783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:04:42.192901 17783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.0.236.84:57441/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:37465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:04:42.193374 17783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:04:42.193693 17783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:04:42.199926 17793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:04:42.200948 17783 server_base.cc:1034] running on GCE node
W20260806 15:04:42.202054 17795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:04:42.205850 17792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:04:42.206176 17783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:04:42.206434 17783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:04:42.209685 17783 hybrid_clock.cc:648] HybridClock initialized: now 1786028682209661 us; error 34 us; skew 500 ppm
Aug 06 15:04:42 dist-test-slave-tc2s krb5kdc[16599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028682, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:04:42.212774 17783 init.cc:377] Logged in from keytab as kudu/127.0.236.67@KRBTEST.COM (short username kudu)
I20260806 15:04:42.213248 17783 webserver.cc:460] Webserver started at http://127.0.236.67:41583/ using document root <none> and password file <none>
I20260806 15:04:42.213464 17783 fs_manager.cc:359] Metadata directory not provided
I20260806 15:04:42.213512 17783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:04:42.213629 17783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 15:04:42 dist-test-slave-tc2s krb5kdc[16599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028682, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for HTTP/127.0.236.84@KRBTEST.COM
I20260806 15:04:42.229717 17783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-2/data/instance:
uuid: "cfc4dd582c89405eb7384bee34a95c10"
format_stamp: "Formatted at 2026-08-06 15:04:42 on dist-test-slave-tc2s"
server_key: "26d998ab4804bf6abe51a3b2846c2243"
server_key_iv: "b6c4a91f1d7c252e5e0258f3d24692fb"
server_key_version: "kuduclusterkey@0"
I20260806 15:04:42.230873 17783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-2/wal/instance:
uuid: "cfc4dd582c89405eb7384bee34a95c10"
format_stamp: "Formatted at 2026-08-06 15:04:42 on dist-test-slave-tc2s"
server_key: "26d998ab4804bf6abe51a3b2846c2243"
server_key_iv: "b6c4a91f1d7c252e5e0258f3d24692fb"
server_key_version: "kuduclusterkey@0"
I20260806 15:04:42.241019 17783 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.001s sys 0.000s
I20260806 15:04:42.259482 17809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:04:42.259686 17783 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260806 15:04:42.259816 17783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "cfc4dd582c89405eb7384bee34a95c10"
format_stamp: "Formatted at 2026-08-06 15:04:42 on dist-test-slave-tc2s"
server_key: "26d998ab4804bf6abe51a3b2846c2243"
server_key_iv: "b6c4a91f1d7c252e5e0258f3d24692fb"
server_key_version: "kuduclusterkey@0"
I20260806 15:04:42.259966 17783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:04:42.316406 17783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:04:42.318931 17783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:04:42.319123 17783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:04:42.319417 17783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:04:42.319810 17783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:04:42.319900 17783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:04:42.319967 17783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:04:42.320020 17783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:04:42.328619 17783 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:34083
I20260806 15:04:42.329070 17783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
I20260806 15:04:42.330307 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 17783
I20260806 15:04:42.330430 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-2/wal/instance
I20260806 15:04:42.330634 945 external_mini_cluster.cc:1428] Setting key 0cf3b281622e9540947b8998ae460869
I20260806 15:04:42.339512 17963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:34083 every 8 connection(s)
Aug 06 15:04:42 dist-test-slave-tc2s krb5kdc[16599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028682, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:04:42.357041 17586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:04:42.339175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:55273 (local address 127.0.236.126:37465)
0806 15:04:42.339221 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 15:04:42.339224 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:04:42.345001 (+ 5777us) server_negotiation.cc:407] Connection header received
0806 15:04:42.345703 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:04:42.345704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:04:42.345718 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:04:42.345752 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:04:42.346642 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.347504 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.348978 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.349284 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.349328 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:04:42.353752 (+ 4424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:04:42.353754 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:04:42.353754 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:04:42.353762 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:04:42.355387 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:04:42.355962 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:04:42.355964 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:04:42.355964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:04:42.356027 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:04:42.356286 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:04:42.356286 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:04:42.356287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:04:42.356449 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:04:42.356498 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 15:04:42.356855 (+ 357us) server_negotiation.cc:299] Negotiation successful
0806 15:04:42.356937 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 15:04:42.357537 17967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:04:42.339170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:37465 (local address 127.0.236.67:55273)
0806 15:04:42.344854 (+ 5684us) negotiation.cc:107] Waiting for socket to connect
0806 15:04:42.344906 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 15:04:42.345575 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:04:42.345974 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:04:42.345979 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:04:42.346245 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:04:42.346536 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:04:42.346543 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.347612 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.347613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:04:42.348878 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:04:42.348881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.349429 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.349430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:04:42.349573 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:04:42.351408 (+ 1835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:04:42.351418 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:04:42.353652 (+ 2234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:04:42.355474 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:04:42.355480 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:04:42.355483 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:04:42.355756 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:04:42.356090 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:04:42.356091 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:04:42.356091 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:04:42.356219 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:04:42.356531 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:04:42.356538 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:04:42.356695 (+ 157us) client_negotiation.cc:769] Sending connection context
0806 15:04:42.357146 (+ 451us) client_negotiation.cc:240] Negotiation successful
0806 15:04:42.357303 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5623,"thread_start_us":149,"threads_started":1}
I20260806 15:04:42.357975 17964 heartbeater.cc:347] Connected to a master server at 127.0.236.126:37465
I20260806 15:04:42.358068 17964 heartbeater.cc:464] Registering TS with master...
I20260806 15:04:42.358248 17964 heartbeater.cc:511] Master 127.0.236.126:37465 requested a full tablet report, sending...
I20260806 15:04:42.358589 17271 ts_manager.cc:194] Registered new tserver with Master: cfc4dd582c89405eb7384bee34a95c10 (127.0.236.67:34083)
I20260806 15:04:42.359071 17271 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.67@KRBTEST.COM'} at 127.0.236.67:55273
I20260806 15:04:42.364068 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:04:42.377415 17586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:04:42.368066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51466 (local address 127.0.236.126:37465)
0806 15:04:42.368107 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 15:04:42.368109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:04:42.369858 (+ 1749us) server_negotiation.cc:407] Connection header received
0806 15:04:42.369979 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:04:42.369980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:04:42.369990 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:04:42.370020 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:04:42.370814 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.371554 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.372639 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.372934 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.372979 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:04:42.374499 (+ 1520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:04:42.374500 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:04:42.374500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:04:42.374507 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:04:42.376135 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:04:42.376561 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:04:42.376562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:04:42.376562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:04:42.376623 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:04:42.376829 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:04:42.376830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:04:42.376831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:04:42.376961 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:04:42.377033 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 15:04:42.377249 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 15:04:42.377323 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 15:04:42.379264 17271 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51466:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:04:42.381652 17271 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:04:42.392493 17983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:04:42.385630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:35319 (local address 127.0.0.1:44398)
0806 15:04:42.387269 (+ 1639us) negotiation.cc:107] Waiting for socket to connect
0806 15:04:42.387277 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 15:04:42.387325 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:04:42.387552 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:04:42.387553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:04:42.387556 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:04:42.387604 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:04:42.387606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.388872 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.388873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:04:42.391271 (+ 2398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:04:42.391273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.392208 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.392209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:04:42.392291 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:04:42.392303 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 15:04:42.392360 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 15:04:42.392417 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1627,"thread_start_us":72,"threads_started":1}
I20260806 15:04:42.393002 17990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:04:42.386027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44398 (local address 127.0.236.65:35319)
0806 15:04:42.386711 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0806 15:04:42.386715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:04:42.387331 (+ 616us) server_negotiation.cc:407] Connection header received
0806 15:04:42.387376 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:04:42.387378 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:04:42.387395 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:04:42.387486 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:04:42.387671 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.388785 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.391340 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.392057 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.392120 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:04:42.392175 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 15:04:42.392882 (+ 707us) server_negotiation.cc:299] Negotiation successful
0806 15:04:42.392929 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":58,"threads_started":1}
I20260806 15:04:42.393694 17989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:04:42.384823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:43245 (local address 127.0.0.1:56810)
0806 15:04:42.386484 (+ 1661us) negotiation.cc:107] Waiting for socket to connect
0806 15:04:42.386523 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 15:04:42.386590 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:04:42.386984 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:04:42.386987 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:04:42.386990 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:04:42.387047 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:04:42.387048 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.388898 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.388900 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:04:42.391135 (+ 2235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:04:42.391138 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.393469 (+ 2331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.393470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:04:42.393535 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:04:42.393538 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:04:42.393586 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 15:04:42.393624 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1635,"mutex_wait_us":1110,"thread_start_us":93,"threads_started":1}
I20260806 15:04:42.393750 17461 tablet_service.cc:1467] Processing CreateTablet for tablet 6c8025514d074136b88999749349c298 (DEFAULT_TABLE table=test-table [id=43cb3391f7864adab8d941be7fe20b52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:04:42.394019 17988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:04:42.385310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56810 (local address 127.0.236.66:43245)
0806 15:04:42.386420 (+ 1110us) server_negotiation.cc:206] Beginning negotiation
0806 15:04:42.386424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:04:42.386738 (+ 314us) server_negotiation.cc:407] Connection header received
0806 15:04:42.386787 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:04:42.386789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:04:42.386805 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:04:42.386893 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:04:42.387398 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.388784 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.392222 (+ 3438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.393295 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.393357 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:04:42.393434 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 15:04:42.393855 (+ 421us) server_negotiation.cc:299] Negotiation successful
0806 15:04:42.393937 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1069,"thread_start_us":83,"threads_started":1}
I20260806 15:04:42.394115 17461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c8025514d074136b88999749349c298. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:04:42.394621 17679 tablet_service.cc:1467] Processing CreateTablet for tablet 6c8025514d074136b88999749349c298 (DEFAULT_TABLE table=test-table [id=43cb3391f7864adab8d941be7fe20b52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:04:42.394985 17679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c8025514d074136b88999749349c298. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:04:42.397804 17991 tablet_bootstrap.cc:492] T 6c8025514d074136b88999749349c298 P 1111f3c7318c4883a2cbb733ca2b4639: Bootstrap starting.
I20260806 15:04:42.398777 17991 tablet_bootstrap.cc:654] T 6c8025514d074136b88999749349c298 P 1111f3c7318c4883a2cbb733ca2b4639: Neither blocks nor log segments found. Creating new log.
I20260806 15:04:42.399140 17991 log.cc:824] T 6c8025514d074136b88999749349c298 P 1111f3c7318c4883a2cbb733ca2b4639: Log is configured to *not* fsync() on all Append() calls
I20260806 15:04:42.399874 17982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:04:42.385088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:34083 (local address 127.0.0.1:55278)
0806 15:04:42.387141 (+ 2053us) negotiation.cc:107] Waiting for socket to connect
0806 15:04:42.387149 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 15:04:42.387184 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:04:42.389351 (+ 2167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:04:42.389352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:04:42.389354 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:04:42.389410 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:04:42.389412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.391016 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.391017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:04:42.392120 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:04:42.392123 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.399633 (+ 7510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.399634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:04:42.399713 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:04:42.399718 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 15:04:42.399752 (+ 34us) client_negotiation.cc:240] Negotiation successful
0806 15:04:42.399792 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2038,"thread_start_us":73,"threads_started":1}
I20260806 15:04:42.400157 17991 tablet_bootstrap.cc:492] T 6c8025514d074136b88999749349c298 P 1111f3c7318c4883a2cbb733ca2b4639: No bootstrap required, opened a new log
I20260806 15:04:42.400213 17991 ts_tablet_manager.cc:1397] T 6c8025514d074136b88999749349c298 P 1111f3c7318c4883a2cbb733ca2b4639: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:04:42.400279 17987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:04:42.385189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55278 (local address 127.0.236.67:34083)
0806 15:04:42.389169 (+ 3980us) server_negotiation.cc:206] Beginning negotiation
0806 15:04:42.389174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:04:42.389185 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:04:42.389220 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:04:42.389231 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:04:42.389247 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:04:42.389485 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:04:42.389556 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.390916 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.398699 (+ 7783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.399481 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.400040 (+ 559us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:04:42.400128 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 15:04:42.400154 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 15:04:42.400191 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3944,"thread_start_us":158,"threads_started":1}
I20260806 15:04:42.402707 17991 raft_consensus.cc:348] T 6c8025514d074136b88999749349c298 P 1111f3c7318c4883a2cbb733ca2b4639 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdc631647c3b48de8f1249ee6fd7e1d9" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 43245 } } peers { permanent_uuid: "1111f3c7318c4883a2cbb733ca2b4639" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 35319 } } peers { permanent_uuid: "cfc4dd582c89405eb7384bee34a95c10" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 34083 } }
I20260806 15:04:42.402861 17991 raft_consensus.cc:374] T 6c8025514d074136b88999749349c298 P 1111f3c7318c4883a2cbb733ca2b4639 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:04:42.402889 17991 raft_consensus.cc:729] T 6c8025514d074136b88999749349c298 P 1111f3c7318c4883a2cbb733ca2b4639 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1111f3c7318c4883a2cbb733ca2b4639, State: Initialized, Role: FOLLOWER
I20260806 15:04:42.403687 17876 tablet_service.cc:1467] Processing CreateTablet for tablet 6c8025514d074136b88999749349c298 (DEFAULT_TABLE table=test-table [id=43cb3391f7864adab8d941be7fe20b52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:04:42.404035 17876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c8025514d074136b88999749349c298. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:04:42.404037 17991 consensus_queue.cc:260] T 6c8025514d074136b88999749349c298 P 1111f3c7318c4883a2cbb733ca2b4639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdc631647c3b48de8f1249ee6fd7e1d9" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 43245 } } peers { permanent_uuid: "1111f3c7318c4883a2cbb733ca2b4639" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 35319 } } peers { permanent_uuid: "cfc4dd582c89405eb7384bee34a95c10" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 34083 } }
I20260806 15:04:42.404536 17991 ts_tablet_manager.cc:1428] T 6c8025514d074136b88999749349c298 P 1111f3c7318c4883a2cbb733ca2b4639: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 15:04:42.404693 17583 heartbeater.cc:503] Master 127.0.236.126:37465 was elected leader, sending a full tablet report...
I20260806 15:04:42.406983 17998 tablet_bootstrap.cc:492] T 6c8025514d074136b88999749349c298 P cfc4dd582c89405eb7384bee34a95c10: Bootstrap starting.
I20260806 15:04:42.407179 17992 tablet_bootstrap.cc:492] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9: Bootstrap starting.
I20260806 15:04:42.408085 17998 tablet_bootstrap.cc:654] T 6c8025514d074136b88999749349c298 P cfc4dd582c89405eb7384bee34a95c10: Neither blocks nor log segments found. Creating new log.
I20260806 15:04:42.408181 17992 tablet_bootstrap.cc:654] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9: Neither blocks nor log segments found. Creating new log.
I20260806 15:04:42.408494 17998 log.cc:824] T 6c8025514d074136b88999749349c298 P cfc4dd582c89405eb7384bee34a95c10: Log is configured to *not* fsync() on all Append() calls
I20260806 15:04:42.408489 17992 log.cc:824] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9: Log is configured to *not* fsync() on all Append() calls
I20260806 15:04:42.409667 17992 tablet_bootstrap.cc:492] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9: No bootstrap required, opened a new log
I20260806 15:04:42.409999 17992 ts_tablet_manager.cc:1397] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 15:04:42.411989 17992 raft_consensus.cc:348] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdc631647c3b48de8f1249ee6fd7e1d9" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 43245 } } peers { permanent_uuid: "1111f3c7318c4883a2cbb733ca2b4639" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 35319 } } peers { permanent_uuid: "cfc4dd582c89405eb7384bee34a95c10" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 34083 } }
I20260806 15:04:42.412501 17998 tablet_bootstrap.cc:492] T 6c8025514d074136b88999749349c298 P cfc4dd582c89405eb7384bee34a95c10: No bootstrap required, opened a new log
I20260806 15:04:42.412609 17998 ts_tablet_manager.cc:1397] T 6c8025514d074136b88999749349c298 P cfc4dd582c89405eb7384bee34a95c10: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260806 15:04:42.414455 17998 raft_consensus.cc:348] T 6c8025514d074136b88999749349c298 P cfc4dd582c89405eb7384bee34a95c10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdc631647c3b48de8f1249ee6fd7e1d9" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 43245 } } peers { permanent_uuid: "1111f3c7318c4883a2cbb733ca2b4639" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 35319 } } peers { permanent_uuid: "cfc4dd582c89405eb7384bee34a95c10" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 34083 } }
I20260806 15:04:42.414669 17998 raft_consensus.cc:374] T 6c8025514d074136b88999749349c298 P cfc4dd582c89405eb7384bee34a95c10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:04:42.414746 17998 raft_consensus.cc:729] T 6c8025514d074136b88999749349c298 P cfc4dd582c89405eb7384bee34a95c10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfc4dd582c89405eb7384bee34a95c10, State: Initialized, Role: FOLLOWER
I20260806 15:04:42.415189 17998 consensus_queue.cc:260] T 6c8025514d074136b88999749349c298 P cfc4dd582c89405eb7384bee34a95c10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdc631647c3b48de8f1249ee6fd7e1d9" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 43245 } } peers { permanent_uuid: "1111f3c7318c4883a2cbb733ca2b4639" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 35319 } } peers { permanent_uuid: "cfc4dd582c89405eb7384bee34a95c10" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 34083 } }
I20260806 15:04:42.415520 17998 ts_tablet_manager.cc:1428] T 6c8025514d074136b88999749349c298 P cfc4dd582c89405eb7384bee34a95c10: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 15:04:42.415658 17964 heartbeater.cc:503] Master 127.0.236.126:37465 was elected leader, sending a full tablet report...
I20260806 15:04:42.416293 17992 raft_consensus.cc:374] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:04:42.416389 17992 raft_consensus.cc:729] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdc631647c3b48de8f1249ee6fd7e1d9, State: Initialized, Role: FOLLOWER
I20260806 15:04:42.416567 17992 consensus_queue.cc:260] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdc631647c3b48de8f1249ee6fd7e1d9" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 43245 } } peers { permanent_uuid: "1111f3c7318c4883a2cbb733ca2b4639" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 35319 } } peers { permanent_uuid: "cfc4dd582c89405eb7384bee34a95c10" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 34083 } }
I20260806 15:04:42.416941 17992 ts_tablet_manager.cc:1428] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260806 15:04:42.417085 17778 heartbeater.cc:503] Master 127.0.236.126:37465 was elected leader, sending a full tablet report...
I20260806 15:04:42.448840 18003 raft_consensus.cc:482] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:04:42.448968 18003 raft_consensus.cc:504] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdc631647c3b48de8f1249ee6fd7e1d9" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 43245 } } peers { permanent_uuid: "1111f3c7318c4883a2cbb733ca2b4639" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 35319 } } peers { permanent_uuid: "cfc4dd582c89405eb7384bee34a95c10" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 34083 } }
I20260806 15:04:42.449312 18003 leader_election.cc:288] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1111f3c7318c4883a2cbb733ca2b4639 (127.0.236.65:35319), cfc4dd582c89405eb7384bee34a95c10 (127.0.236.67:34083)
I20260806 15:04:42.453642 17781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:04:42.449571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:35319 (local address 127.0.236.66:42411)
0806 15:04:42.449841 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 15:04:42.449852 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:04:42.449918 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:04:42.450533 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:04:42.450534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:04:42.450536 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:04:42.450603 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:04:42.450605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.451518 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.451519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:04:42.452668 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:04:42.452671 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.453418 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.453419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:04:42.453491 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:04:42.453495 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:04:42.453536 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 15:04:42.453574 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251}
I20260806 15:04:42.454300 17990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:04:42.449999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:42411 (local address 127.0.236.65:35319)
0806 15:04:42.450042 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 15:04:42.450043 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:04:42.450050 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:04:42.450075 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:04:42.450076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:04:42.450084 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:04:42.450122 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:04:42.450684 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.451422 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.452764 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.453301 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.453365 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:04:42.453374 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 15:04:42.454193 (+ 819us) server_negotiation.cc:299] Negotiation successful
0806 15:04:42.454237 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 15:04:42.454545 17501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c8025514d074136b88999749349c298" candidate_uuid: "cdc631647c3b48de8f1249ee6fd7e1d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1111f3c7318c4883a2cbb733ca2b4639" is_pre_election: true
I20260806 15:04:42.454674 17501 raft_consensus.cc:2454] T 6c8025514d074136b88999749349c298 P 1111f3c7318c4883a2cbb733ca2b4639 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdc631647c3b48de8f1249ee6fd7e1d9 in term 0.
I20260806 15:04:42.454948 17622 leader_election.cc:302] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1111f3c7318c4883a2cbb733ca2b4639, cdc631647c3b48de8f1249ee6fd7e1d9; no voters:
I20260806 15:04:42.455217 18003 raft_consensus.cc:2781] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:04:42.455281 18003 raft_consensus.cc:482] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:04:42.455302 18003 raft_consensus.cc:3037] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:04:42.456462 18003 raft_consensus.cc:504] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdc631647c3b48de8f1249ee6fd7e1d9" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 43245 } } peers { permanent_uuid: "1111f3c7318c4883a2cbb733ca2b4639" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 35319 } } peers { permanent_uuid: "cfc4dd582c89405eb7384bee34a95c10" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 34083 } }
I20260806 15:04:42.456585 18003 leader_election.cc:288] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9 [CANDIDATE]: Term 1 election: Requested vote from peers 1111f3c7318c4883a2cbb733ca2b4639 (127.0.236.65:35319), cfc4dd582c89405eb7384bee34a95c10 (127.0.236.67:34083)
I20260806 15:04:42.456825 17501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c8025514d074136b88999749349c298" candidate_uuid: "cdc631647c3b48de8f1249ee6fd7e1d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1111f3c7318c4883a2cbb733ca2b4639"
I20260806 15:04:42.456904 17501 raft_consensus.cc:3037] T 6c8025514d074136b88999749349c298 P 1111f3c7318c4883a2cbb733ca2b4639 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:04:42.457746 17501 raft_consensus.cc:2454] T 6c8025514d074136b88999749349c298 P 1111f3c7318c4883a2cbb733ca2b4639 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdc631647c3b48de8f1249ee6fd7e1d9 in term 1.
I20260806 15:04:42.458001 17622 leader_election.cc:302] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1111f3c7318c4883a2cbb733ca2b4639, cdc631647c3b48de8f1249ee6fd7e1d9; no voters:
I20260806 15:04:42.458115 18003 raft_consensus.cc:2781] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:04:42.458359 18003 raft_consensus.cc:686] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9 [term 1 LEADER]: Becoming Leader. State: Replica: cdc631647c3b48de8f1249ee6fd7e1d9, State: Running, Role: LEADER
I20260806 15:04:42.458470 18003 consensus_queue.cc:237] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdc631647c3b48de8f1249ee6fd7e1d9" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 43245 } } peers { permanent_uuid: "1111f3c7318c4883a2cbb733ca2b4639" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 35319 } } peers { permanent_uuid: "cfc4dd582c89405eb7384bee34a95c10" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 34083 } }
I20260806 15:04:42.459405 17271 catalog_manager.cc:5495] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdc631647c3b48de8f1249ee6fd7e1d9 (127.0.236.66). New cstate: current_term: 1 leader_uuid: "cdc631647c3b48de8f1249ee6fd7e1d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdc631647c3b48de8f1249ee6fd7e1d9" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 43245 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1111f3c7318c4883a2cbb733ca2b4639" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 35319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cfc4dd582c89405eb7384bee34a95c10" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 34083 } health_report { overall_health: UNKNOWN } } }
I20260806 15:04:42.463196 17781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:04:42.450253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:34083 (local address 127.0.236.66:53569)
0806 15:04:42.453706 (+ 3453us) negotiation.cc:107] Waiting for socket to connect
0806 15:04:42.453714 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 15:04:42.453746 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:04:42.460263 (+ 6517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:04:42.460265 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:04:42.460267 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:04:42.460301 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:04:42.460303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.461195 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.461196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:04:42.462246 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:04:42.462249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.462994 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.462995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:04:42.463054 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:04:42.463057 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:04:42.463098 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 15:04:42.463130 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3443,"thread_start_us":109,"threads_started":1}
I20260806 15:04:42.463506 17987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:04:42.450461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:53569 (local address 127.0.236.67:34083)
0806 15:04:42.450500 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 15:04:42.450502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:04:42.453792 (+ 3290us) server_negotiation.cc:407] Connection header received
0806 15:04:42.453818 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:04:42.453819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:04:42.453828 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:04:42.453865 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:04:42.460408 (+ 6543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.461081 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.462340 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.462883 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.462953 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:04:42.462962 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 15:04:42.463401 (+ 439us) server_negotiation.cc:299] Negotiation successful
0806 15:04:42.463443 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:04:42.464010 17904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c8025514d074136b88999749349c298" candidate_uuid: "cdc631647c3b48de8f1249ee6fd7e1d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfc4dd582c89405eb7384bee34a95c10" is_pre_election: true
I20260806 15:04:42.464162 17904 raft_consensus.cc:2454] T 6c8025514d074136b88999749349c298 P cfc4dd582c89405eb7384bee34a95c10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdc631647c3b48de8f1249ee6fd7e1d9 in term 0.
I20260806 15:04:42.464288 17903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c8025514d074136b88999749349c298" candidate_uuid: "cdc631647c3b48de8f1249ee6fd7e1d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfc4dd582c89405eb7384bee34a95c10"
I20260806 15:04:42.464377 17903 raft_consensus.cc:3037] T 6c8025514d074136b88999749349c298 P cfc4dd582c89405eb7384bee34a95c10 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:04:42.465227 17903 raft_consensus.cc:2454] T 6c8025514d074136b88999749349c298 P cfc4dd582c89405eb7384bee34a95c10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdc631647c3b48de8f1249ee6fd7e1d9 in term 1.
I20260806 15:04:42.489739 17988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:04:42.486281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56822 (local address 127.0.236.66:43245)
0806 15:04:42.486341 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 15:04:42.486342 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:04:42.486361 (+ 19us) server_negotiation.cc:407] Connection header received
0806 15:04:42.486402 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:04:42.486403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:04:42.486427 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:04:42.486506 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:04:42.486701 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.487398 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.488572 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:04:42.488876 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:04:42.488951 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:04:42.489169 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:04:42.489304 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:04:42.489388 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 15:04:42.489621 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 15:04:42.489642 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 15:04:42.491822 17501 raft_consensus.cc:1261] T 6c8025514d074136b88999749349c298 P 1111f3c7318c4883a2cbb733ca2b4639 [term 1 FOLLOWER]: Refusing update from remote peer cdc631647c3b48de8f1249ee6fd7e1d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:04:42.491822 17903 raft_consensus.cc:1261] T 6c8025514d074136b88999749349c298 P cfc4dd582c89405eb7384bee34a95c10 [term 1 FOLLOWER]: Refusing update from remote peer cdc631647c3b48de8f1249ee6fd7e1d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:04:42.492192 18003 consensus_queue.cc:1035] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1111f3c7318c4883a2cbb733ca2b4639" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 35319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:04:42.492353 18003 consensus_queue.cc:1035] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfc4dd582c89405eb7384bee34a95c10" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 34083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:04:42.497809 17271 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:04:42.497937 17271 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:04:42.498708 17271 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed30a8de21fb480da1d6dc2e6ec082a9: Sending DeleteTablet for 3 replicas of tablet 6c8025514d074136b88999749349c298
I20260806 15:04:42.499269 17679 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c8025514d074136b88999749349c298 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:04:42 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:56810
I20260806 15:04:42.499555 17461 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c8025514d074136b88999749349c298 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:04:42 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:44398
I20260806 15:04:42.498291 18019 mvcc.cc:205] Tried to move back new op lower bound from 7315573483483602944 to 7315573483350618112. Current Snapshot: MvccSnapshot[applied={T|T < 7315573483483602944}]
I20260806 15:04:42.499702 18026 tablet_replica.cc:330] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9: stopping tablet replica
I20260806 15:04:42.499797 18026 raft_consensus.cc:2229] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9 [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:04:42.499892 18026 raft_consensus.cc:2258] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:04:42.499963 17876 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c8025514d074136b88999749349c298 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:04:42 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:55278
I20260806 15:04:42.500170 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 17386
Aug 06 15:04:42 dist-test-slave-tc2s krb5kdc[16599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028681, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for HTTP/127.0.236.84@KRBTEST.COM
I20260806 15:04:42.516149 18028 tablet_replica.cc:330] T 6c8025514d074136b88999749349c298 P cfc4dd582c89405eb7384bee34a95c10: stopping tablet replica
I20260806 15:04:42.516297 18028 raft_consensus.cc:2229] T 6c8025514d074136b88999749349c298 P cfc4dd582c89405eb7384bee34a95c10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:04:42.516381 18028 pending_rounds.cc:62] T 6c8025514d074136b88999749349c298 P cfc4dd582c89405eb7384bee34a95c10: Trying to abort 2 pending ops.
I20260806 15:04:42.516430 18028 pending_rounds.cc:68] T 6c8025514d074136b88999749349c298 P cfc4dd582c89405eb7384bee34a95c10: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7315573483483602944 op_type: WRITE_OP write_request { tablet_id: "6c8025514d074136b88999749349c298" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\266\307\322\323\006\"8\n\ntest-admin\022*\n 43cb3391f7864adab8d941be7fe20b52\020\001\030\001 \001(\001" signature: "<redacted>""\223\276\346\020\022@N9\261\016R\210\354vQ\365\025T\037\242|x\214\031\003Gc\2346\032u\354C\231mt\272\027\360\307=\334F\310p\215t0#f\207\351\245\321\002\256\326\372A\203R\341\344\277" signing_key_seq_num: 0 } } request_id { client_id: "146b3f9d20f9463b8342e31172272dd1" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260806 15:04:42.516669 18028 pending_rounds.cc:68] T 6c8025514d074136b88999749349c298 P cfc4dd582c89405eb7384bee34a95c10: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315573483350618112 op_type: NO_OP noop_request { }
I20260806 15:04:42.516724 18028 raft_consensus.cc:2866] T 6c8025514d074136b88999749349c298 P cfc4dd582c89405eb7384bee34a95c10 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260806 15:04:42.516758 18028 raft_consensus.cc:2258] T 6c8025514d074136b88999749349c298 P cfc4dd582c89405eb7384bee34a95c10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:04:42.521090 18026 ts_tablet_manager.cc:1905] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:04:42.522763 18026 ts_tablet_manager.cc:1918] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:04:42.522843 18026 log.cc:1196] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-1/wal/wals/6c8025514d074136b88999749349c298
I20260806 15:04:42.523089 18026 ts_tablet_manager.cc:1939] T 6c8025514d074136b88999749349c298 P cdc631647c3b48de8f1249ee6fd7e1d9: Deleting consensus metadata
I20260806 15:04:42.523509 17249 catalog_manager.cc:4841] TS cdc631647c3b48de8f1249ee6fd7e1d9 (127.0.236.66:43245): tablet 6c8025514d074136b88999749349c298 (table test-table [id=43cb3391f7864adab8d941be7fe20b52]) successfully deleted
W20260806 15:04:42.530154 18021 consensus_queue.cc:1159] T 6c8025514d074136b88999749349c298 P cfc4dd582c89405eb7384bee34a95c10 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cfc4dd582c89405eb7384bee34a95c10" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
W20260806 15:04:42.530630 17250 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.236.65:35319 (error 108)
I20260806 15:04:42.530831 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 17592
I20260806 15:04:42.535843 18028 ts_tablet_manager.cc:1905] T 6c8025514d074136b88999749349c298 P cfc4dd582c89405eb7384bee34a95c10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:04:42.537644 18028 ts_tablet_manager.cc:1918] T 6c8025514d074136b88999749349c298 P cfc4dd582c89405eb7384bee34a95c10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:04:42.537773 18028 log.cc:1196] T 6c8025514d074136b88999749349c298 P cfc4dd582c89405eb7384bee34a95c10: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/6c8025514d074136b88999749349c298
I20260806 15:04:42.538120 18028 ts_tablet_manager.cc:1939] T 6c8025514d074136b88999749349c298 P cfc4dd582c89405eb7384bee34a95c10: Deleting consensus metadata
I20260806 15:04:42.538640 17251 catalog_manager.cc:4841] TS cfc4dd582c89405eb7384bee34a95c10 (127.0.236.67:34083): tablet 6c8025514d074136b88999749349c298 (table test-table [id=43cb3391f7864adab8d941be7fe20b52]) successfully deleted
I20260806 15:04:42.539636 17982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:04:42.539512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:35319 (local address 127.0.0.1:44402)
0806 15:04:42.539558 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 15:04:42.539601 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.65:35319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 15:04:42.539837 17250 catalog_manager.cc:4584] TS 1111f3c7318c4883a2cbb733ca2b4639 (127.0.236.65:35319): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6c8025514d074136b88999749349c298: Network error: Client connection negotiation failed: client connection to 127.0.236.65:35319: connect: Connection refused (error 111)
I20260806 15:04:42.548621 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 17783
I20260806 15:04:42.564882 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 17226
I20260806 15:04:42.575564 945 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 15:04:42.919003 945 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 15:04:42.919093 945 mini_ranger.cc:66] Stopping Ranger...
I20260806 15:04:43.262274 945 mini_ranger.cc:68] Stopped Ranger
2026-08-06 15:04:43.262 UTC [16636] LOG: received smart shutdown request
2026-08-06 15:04:43.265 UTC [16636] LOG: background worker "logical replication launcher" (PID 16646) exited with exit code 1
2026-08-06 15:04:43.266 UTC [16641] LOG: shutting down
2026-08-06 15:04:43.382 UTC [16636] LOG: database system is shut down
2026-08-06T15:04:43Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (43839 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:04:43 dist-test-slave-tc2s krb5kdc[18046](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:04:43 dist-test-slave-tc2s krb5kdc[18046](info): set up 2 sockets
Aug 06 15:04:43 dist-test-slave-tc2s krb5kdc[18046](info): commencing operation
krb5kdc: starting...
W20260806 15:04:44.574237 945 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:04:44 dist-test-slave-tc2s krb5kdc[18046](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:04:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:04:44Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.0.236.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.0.236.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.0.236.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/rangeradmin_127.0.236.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.0.236.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/rangeradmin_127.0.236.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.0.236.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.0.236.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.0.236.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/rangerlookup_127.0.236.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.0.236.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/rangerlookup_127.0.236.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.236.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.0.236.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.236.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/HTTP_127.0.236.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.236.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/HTTP_127.0.236.84@KRBTEST.COM.keytab.
I20260806 15:04:44.719942 945 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskFIWuwX/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/postgres -l logfile start
2026-08-06 15:04:46.792 UTC [18099] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 15:04:46.792 UTC [18099] LOG: listening on IPv4 address "127.0.236.84", port 46529
2026-08-06 15:04:46.794 UTC [18099] LOG: listening on Unix socket "/tmp/.s.PGSQL.46529"
2026-08-06 15:04:46.813 UTC [18104] LOG: database system was shut down at 2026-08-06 15:04:45 UTC
2026-08-06 15:04:46.817 UTC [18099] LOG: database system is ready to accept connections
I20260806 15:04:47.856338 945 mini_postgres.cc:96] Postgres bound to 46529
2026-08-06 15:04:47.862 UTC [18121] FATAL: database "slave" does not exist
127.0.236.84:46529 - accepting connections
I20260806 15:04:47.863397 945 mini_ranger.cc:161] Starting Ranger...
I20260806 15:04:47.877478 945 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 15:04:48.303416 945 mini_ranger.cc:87] Created ranger Postgres database
I20260806 15:04:48.303498 945 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ranger-admin
2026-08-06 15:04:48,571 [I] DB FLAVOR :POSTGRES
2026-08-06 15:04:48,571 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 15:04:48,571 [I] Checking connection..
2026-08-06 15:04:48,571 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:46529/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 15:04:48,822 [I] Checking connection passed.
2026-08-06 15:04:48,822 [I] --------- Verifying version history table ---------
2026-08-06 15:04:48,823 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:46529/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 15:04:49,071 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 15:04:49,071 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:46529/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 15:04:49,320 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 15:04:49,320 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 15:04:49,320 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:46529/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 15:04:49.566 UTC [18228] WARNING: there is no transaction in progress
2026-08-06 15:04:49,572 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 15:04:49,573 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:46529/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 15:04:49,821 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 15:04:49,821 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 15:04:49,821 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:46529/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 15:04:50,043 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:46529/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 15:04:50,285 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:46529/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-tc2s.c.gcp-upstream.internal','N') ;"
2026-08-06 15:04:50,503 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:46529/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 15:04:50,756 [I] Table x_portal_user does not exist in database ranger
2026-08-06 15:04:50,756 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:46529/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 15:04:51,004 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 15:04:51,004 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:46529/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 15:04:51,246 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:46529/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 15:04:51,491 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 15:04:51,491 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:46529/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 15:04:52.318 UTC [18402] WARNING: there is no transaction in progress
2026-08-06 15:04:52.332 UTC [18402] WARNING: there is no transaction in progress
2026-08-06 15:04:52.346 UTC [18402] WARNING: there is no transaction in progress
2026-08-06 15:04:52.358 UTC [18402] WARNING: there is no transaction in progress
2026-08-06 15:04:52.372 UTC [18402] WARNING: there is no transaction in progress
2026-08-06 15:04:52.386 UTC [18402] WARNING: there is no transaction in progress
2026-08-06 15:04:52.435 UTC [18402] WARNING: there is no transaction in progress
2026-08-06 15:04:52.441 UTC [18402] WARNING: there is no transaction in progress
2026-08-06 15:04:52.453 UTC [18402] WARNING: there is no transaction in progress
2026-08-06 15:04:52.461 UTC [18402] WARNING: there is no transaction in progress
2026-08-06 15:04:52.470 UTC [18402] WARNING: there is no transaction in progress
2026-08-06 15:04:52.482 UTC [18402] WARNING: there is no transaction in progress
2026-08-06 15:04:52.490 UTC [18402] WARNING: there is no transaction in progress
2026-08-06 15:04:52.496 UTC [18402] WARNING: there is no transaction in progress
2026-08-06 15:04:52.501 UTC [18402] WARNING: there is no transaction in progress
2026-08-06 15:04:52.507 UTC [18402] WARNING: there is no transaction in progress
2026-08-06 15:04:53,181 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 15:04:53,182 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:46529/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 15:04:53,630 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 15:04:53,631 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:46529/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 15:04:53,894 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 15:04:53,894 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:46529/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 15:04:54,147 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 15:04:54,148 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:46529/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 15:04:54,392 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:46529/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 15:04:54,640 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:46529/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-tc2s.c.gcp-upstream.internal';"
2026-08-06 15:04:54,878 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 15:04:54,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:46529/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 15:04:55,124 [I] DB_PATCHES have already been applied
I20260806 15:04:55.129546 945 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ranger-admin:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskFIWuwX/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 15:04:55.129606 945 mini_ranger.cc:193] Using host: 127.0.236.84
I20260806 15:04:55.133472 945 mini_ranger.cc:227] Ranger admin URL: http://127.0.236.84:40671
Aug 06, 2026 3:04:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 3:04:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 3:04:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 3:04:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 3:04:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 3:04:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 3:04:55 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 15:04:56.331216 945 mini_ranger.cc:160] Time spent starting Ranger: real 8.468s user 0.000s sys 0.009s
Aug 06, 2026 3:05:14 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 3:05:14 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 3:05:14 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 3:05:14 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 3:05:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 3:05:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 3:05:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 3:05:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 3:05:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 3:05:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 3:05:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 3:05:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 3:05:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 3:05:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 3:05:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 3:05:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 3:05:14 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 3:05:15 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 15:05:15.625480 945 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.0.236.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.0.236.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.0.236.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/rangerkms_127.0.236.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.0.236.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/rangerkms_127.0.236.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.236.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.0.236.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.236.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/HTTP_127.0.236.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.236.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/HTTP_127.0.236.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 15:05:15 dist-test-slave-tc2s krb5kdc[18046](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028715, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 15:05:15.699982 945 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 15:05:15.708451 945 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 15:05:16.356146 945 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 15:05:16.356231 945 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ranger-kms
I20260806 15:05:16.356247 945 mini_ranger_kms.cc:227] Using postgres at 127.0.236.84:46529
2026-08-06 15:05:16,503 [I] DB FLAVOR :POSTGRES
2026-08-06 15:05:16,504 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 15:05:16,504 [I] Checking connection
2026-08-06 15:05:16,504 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:46529/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 15:05:16,727 [I] connection success
2026-08-06 15:05:16,728 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 15:05:16,728 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 15:05:16,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:46529/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 15:05:16,953 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 15:05:16,953 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 15:05:16,953 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 15:05:16,953 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.236.84:46529/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 15:05:17,224 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 15:05:17.668762 945 mini_ranger_kms.cc:310] Created kms service
I20260806 15:05:17.840940 945 mini_ranger_kms.cc:326] Created kudu user
I20260806 15:05:17.882670 945 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 15:05:17 dist-test-slave-tc2s krb5kdc[18046](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028715, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.0.236.84@KRBTEST.COM
I20260806 15:05:18.253269 945 mini_ranger_kms.cc:384] Added ranger policy
I20260806 15:05:18.253378 945 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ranger-kms:/tmp/dist-test-taskFIWuwX/build/release/bin/postgresql.jar:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskFIWuwX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 15:05:18.253396 945 mini_ranger_kms.cc:240] Using host: 127.0.236.84
I20260806 15:05:18.257225 945 mini_ranger_kms.cc:276] Ranger KMS PID: 18661
I20260806 15:05:18.257297 945 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.0.236.84:40643
Aug 06, 2026 3:05:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 3:05:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 3:05:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 3:05:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 3:05:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 3:05:19 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 15:05:20 dist-test-slave-tc2s krb5kdc[18046](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786028720, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.236.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:20.630496 945 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.931s user 0.001s sys 0.130s
I20260806 15:05:20.630599 945 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 15:05:20.630645 945 mini_ranger_kms.cc:401] 127.0.236.84:40643/kms/v1/keys
Aug 06 15:05:20 dist-test-slave-tc2s krb5kdc[18046](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028720, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.236.84@KRBTEST.COM for HTTP/127.0.236.84@KRBTEST.COM
Aug 06 15:05:21 dist-test-slave-tc2s krb5kdc[18046](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028720, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.236.84@KRBTEST.COM for HTTP/127.0.236.84@KRBTEST.COM
Aug 06 15:05:23 dist-test-slave-tc2s krb5kdc[18046](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:05:23.362596 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:38613
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:44515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:38613
--ranger_config_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.236.84:40643/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:05:23.442260 18739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:23.442451 18739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:23.442480 18739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:23.443554 18739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:05:23.443589 18739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:23.443608 18739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:23.443624 18739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:05:23.443638 18739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:05:23.445140 18739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:44515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/wal
--ranger_kms_url=127.0.236.84:40643/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:38613
--ranger_config_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:38613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:23.445363 18739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:23.445554 18739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:05:23.450120 18748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:23.450127 18747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:23.450204 18739 server_base.cc:1034] running on GCE node
W20260806 15:05:23.450127 18750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:23.450601 18739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:23.450858 18739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:23.451992 18739 hybrid_clock.cc:648] HybridClock initialized: now 1786028723451979 us; error 29 us; skew 500 ppm
Aug 06 15:05:23 dist-test-slave-tc2s krb5kdc[18046](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028723, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:23.458034 18739 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:05:23.458410 18739 webserver.cc:460] Webserver started at http://127.0.236.126:34653/ using document root <none> and password file <none>
I20260806 15:05:23.458599 18739 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:23.458639 18739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:05:23.458755 18739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 15:05:23 dist-test-slave-tc2s krb5kdc[18046](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028723, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for HTTP/127.0.236.84@KRBTEST.COM
I20260806 15:05:23.510021 18739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "dbb6d46f3c74487aa3ff246669bfd950"
format_stamp: "Formatted at 2026-08-06 15:05:23 on dist-test-slave-tc2s"
server_key: "a8fb65e87c8ca89ecf3b3f17604aac99"
server_key_iv: "2cfa2accc02c1abee6ed7e6a6a62f779"
server_key_version: "kuduclusterkey@0"
I20260806 15:05:23.510478 18739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "dbb6d46f3c74487aa3ff246669bfd950"
format_stamp: "Formatted at 2026-08-06 15:05:23 on dist-test-slave-tc2s"
server_key: "a8fb65e87c8ca89ecf3b3f17604aac99"
server_key_iv: "2cfa2accc02c1abee6ed7e6a6a62f779"
server_key_version: "kuduclusterkey@0"
I20260806 15:05:23.512352 18739 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 15:05:23.522835 18760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:23.523033 18739 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:05:23.523131 18739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "dbb6d46f3c74487aa3ff246669bfd950"
format_stamp: "Formatted at 2026-08-06 15:05:23 on dist-test-slave-tc2s"
server_key: "a8fb65e87c8ca89ecf3b3f17604aac99"
server_key_iv: "2cfa2accc02c1abee6ed7e6a6a62f779"
server_key_version: "kuduclusterkey@0"
I20260806 15:05:23.523212 18739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:23.538884 18739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:23.540598 18739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:23.540729 18739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:23.544515 18739 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:38613
I20260806 15:05:23.544559 18841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:38613 every 8 connection(s)
I20260806 15:05:23.544863 18739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:05:23.545490 18844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:05:23.547065 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 18739
I20260806 15:05:23.547163 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:05:23.547324 945 external_mini_cluster.cc:1428] Setting key 82d14fc256a682b4e511153d4a6086b3
I20260806 15:05:23.547837 18844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950: Bootstrap starting.
I20260806 15:05:23.548478 18844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950: Neither blocks nor log segments found. Creating new log.
I20260806 15:05:23.548804 18844 log.cc:824] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950: Log is configured to *not* fsync() on all Append() calls
I20260806 15:05:23.549496 18844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950: No bootstrap required, opened a new log
I20260806 15:05:23.551617 18844 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb6d46f3c74487aa3ff246669bfd950" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 38613 } }
I20260806 15:05:23.551771 18844 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:05:23.551810 18844 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbb6d46f3c74487aa3ff246669bfd950, State: Initialized, Role: FOLLOWER
I20260806 15:05:23.551936 18844 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb6d46f3c74487aa3ff246669bfd950" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 38613 } }
I20260806 15:05:23.552011 18844 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:05:23.552032 18844 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:05:23.552083 18844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:05:23.552614 18844 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb6d46f3c74487aa3ff246669bfd950" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 38613 } }
I20260806 15:05:23.552754 18844 leader_election.cc:302] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbb6d46f3c74487aa3ff246669bfd950; no voters:
I20260806 15:05:23.552924 18844 leader_election.cc:288] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:05:23.552990 18852 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:05:23.553251 18844 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:05:23.553251 18852 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [term 1 LEADER]: Becoming Leader. State: Replica: dbb6d46f3c74487aa3ff246669bfd950, State: Running, Role: LEADER
I20260806 15:05:23.553431 18852 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb6d46f3c74487aa3ff246669bfd950" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 38613 } }
Aug 06 15:05:23 dist-test-slave-tc2s krb5kdc[18046](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:05:23.553756 18854 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbb6d46f3c74487aa3ff246669bfd950. Latest consensus state: current_term: 1 leader_uuid: "dbb6d46f3c74487aa3ff246669bfd950" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb6d46f3c74487aa3ff246669bfd950" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 38613 } } }
I20260806 15:05:23.553956 18854 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [sys.catalog]: This master's current role is: LEADER
I20260806 15:05:23.554185 18853 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbb6d46f3c74487aa3ff246669bfd950" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb6d46f3c74487aa3ff246669bfd950" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 38613 } } }
I20260806 15:05:23.554304 18853 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [sys.catalog]: This master's current role is: LEADER
I20260806 15:05:23.554615 18856 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:05:23.555078 18856 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:05:23.555580 18844 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 15:05:23.556595 18856 catalog_manager.cc:1347] Generated new cluster ID: 42b741347bb14033ad8b1be9794ef245
I20260806 15:05:23.556643 18856 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:05:23.557256 18847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:23.547774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36306 (local address 127.0.236.126:38613)
0806 15:05:23.547921 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:23.547929 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:23.547961 (+ 32us) server_negotiation.cc:407] Connection header received
0806 15:05:23.548371 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:23.548377 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:23.548420 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:23.548528 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:23.549292 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:23.550210 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:23.551273 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:23.551641 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:23.551708 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:23.554252 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:23.554257 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:23.554257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:23.554269 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:23.555875 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:23.556274 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:23.556278 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:23.556278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:23.556378 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:23.556563 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:23.556564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:23.556564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:23.556815 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:23.556909 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:23.557036 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 15:05:23.557121 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260806 15:05:23.587674 18856 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:05:23.588440 18856 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:05:23.599541 18856 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950: Generated new TSK 0
I20260806 15:05:23.599699 18856 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.236.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
I20260806 15:05:24.204480 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.236.84:40643/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:0
--local_ip_for_outbound_sockets=127.0.236.65
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:38613
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:44515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:05:24.305639 18911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:24.305819 18911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:24.305840 18911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:24.306947 18911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:24.306985 18911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:24.307009 18911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:05:24.308707 18911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:44515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.0.236.84:40643/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:38613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:24.308946 18911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:24.309226 18911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:05:24.315229 18921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:24.315237 18918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:24.315243 18917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:24.315742 18911 server_base.cc:1034] running on GCE node
I20260806 15:05:24.315958 18911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:24.316215 18911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:24.317395 18911 hybrid_clock.cc:648] HybridClock initialized: now 1786028724317376 us; error 31 us; skew 500 ppm
Aug 06 15:05:24 dist-test-slave-tc2s krb5kdc[18046](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028724, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:24.321930 18911 init.cc:377] Logged in from keytab as kudu/127.0.236.65@KRBTEST.COM (short username kudu)
I20260806 15:05:24.322525 18911 webserver.cc:460] Webserver started at http://127.0.236.65:36943/ using document root <none> and password file <none>
I20260806 15:05:24.322849 18911 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:24.322980 18911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:05:24.323271 18911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 15:05:24 dist-test-slave-tc2s krb5kdc[18046](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028724, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for HTTP/127.0.236.84@KRBTEST.COM
I20260806 15:05:24.338891 18911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/data/instance:
uuid: "14343bc961be4363822f94cce5a6fc5c"
format_stamp: "Formatted at 2026-08-06 15:05:24 on dist-test-slave-tc2s"
server_key: "3c6285e2461dca11ba3cdaffd68f2fc0"
server_key_iv: "e9ba80c602c297c261fe980502c8fef7"
server_key_version: "kuduclusterkey@0"
I20260806 15:05:24.339532 18911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/wal/instance:
uuid: "14343bc961be4363822f94cce5a6fc5c"
format_stamp: "Formatted at 2026-08-06 15:05:24 on dist-test-slave-tc2s"
server_key: "3c6285e2461dca11ba3cdaffd68f2fc0"
server_key_iv: "e9ba80c602c297c261fe980502c8fef7"
server_key_version: "kuduclusterkey@0"
I20260806 15:05:24.342054 18911 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260806 15:05:24.357993 18933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:24.358182 18911 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260806 15:05:24.358300 18911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "14343bc961be4363822f94cce5a6fc5c"
format_stamp: "Formatted at 2026-08-06 15:05:24 on dist-test-slave-tc2s"
server_key: "3c6285e2461dca11ba3cdaffd68f2fc0"
server_key_iv: "e9ba80c602c297c261fe980502c8fef7"
server_key_version: "kuduclusterkey@0"
I20260806 15:05:24.358440 18911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:24.375737 18911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:24.377437 18911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:24.377564 18911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:24.377733 18911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:05:24.378060 18911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:05:24.378098 18911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:24.378123 18911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:05:24.378149 18911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:24.384662 18911 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:37921
I20260806 15:05:24.385066 18911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:05:24.385581 19104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:37921 every 8 connection(s)
Aug 06 15:05:24 dist-test-slave-tc2s krb5kdc[18046](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028724, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:05:24.392575 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 18911
I20260806 15:05:24.392692 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/wal/instance
I20260806 15:05:24.392863 945 external_mini_cluster.cc:1428] Setting key 1648afc86c37e03b9016f0d5fca505ea
I20260806 15:05:24.400110 19108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:24.385768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:38613 (local address 127.0.236.65:48705)
0806 15:05:24.386267 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:24.386312 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:24.386910 (+ 598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:24.387191 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:24.387195 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:24.387418 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:24.387677 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:24.387683 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:24.388674 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:24.388675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:24.389940 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:24.389944 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:24.390464 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:24.390465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:24.390605 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:24.391109 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:24.391119 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:24.395655 (+ 4536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:24.397562 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:24.397570 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:24.397572 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:24.397840 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:24.398055 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:24.398056 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:24.398056 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:24.398179 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:24.398464 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:24.398470 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:24.398623 (+ 153us) client_negotiation.cc:769] Sending connection context
0806 15:05:24.398716 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 15:05:24.398856 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":88,"threads_started":1}
I20260806 15:05:24.400825 19109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:24.386095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:48705 (local address 127.0.236.126:38613)
0806 15:05:24.386244 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:24.386246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:24.386399 (+ 153us) server_negotiation.cc:407] Connection header received
0806 15:05:24.387020 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:24.387022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:24.387035 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:24.387123 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:24.387778 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:24.388591 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:24.390019 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:24.390340 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:24.390408 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:24.395762 (+ 5354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:24.395764 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:24.395764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:24.395772 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:24.397472 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:24.397934 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:24.397935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:24.397936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:24.397996 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:24.398246 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:24.398247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:24.398247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:24.398373 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:24.398427 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:24.400624 (+ 2197us) server_negotiation.cc:299] Negotiation successful
0806 15:05:24.400717 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260806 15:05:24.401340 19105 heartbeater.cc:347] Connected to a master server at 127.0.236.126:38613
I20260806 15:05:24.401430 19105 heartbeater.cc:464] Registering TS with master...
I20260806 15:05:24.401625 19105 heartbeater.cc:511] Master 127.0.236.126:38613 requested a full tablet report, sending...
I20260806 15:05:24.402212 18785 ts_manager.cc:194] Registered new tserver with Master: 14343bc961be4363822f94cce5a6fc5c (127.0.236.65:37921)
I20260806 15:05:24.403072 18785 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.65@KRBTEST.COM'} at 127.0.236.65:48705
WARNING: no policy specified for kudu/127.0.236.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
I20260806 15:05:24.430270 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.236.84:40643/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:0
--local_ip_for_outbound_sockets=127.0.236.66
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:38613
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:44515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 15:05:24 dist-test-slave-tc2s krb5kdc[18046](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786028724, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 15:05:24.556803 19115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:24.557076 19115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:24.557114 19115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:24.558771 19115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:24.558830 19115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:24.558867 19115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:05:24.560971 19115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:44515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.0.236.84:40643/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:38613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:24.561192 19115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:24.561439 19115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:05:24.570227 19124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:24.570223 19125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:24.570516 19127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:24.571358 19115 server_base.cc:1034] running on GCE node
I20260806 15:05:24.571556 19115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:24.571810 19115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:24.572990 19115 hybrid_clock.cc:648] HybridClock initialized: now 1786028724572969 us; error 35 us; skew 500 ppm
Aug 06 15:05:24 dist-test-slave-tc2s krb5kdc[18046](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028724, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:24.576215 19115 init.cc:377] Logged in from keytab as kudu/127.0.236.66@KRBTEST.COM (short username kudu)
I20260806 15:05:24.576687 19115 webserver.cc:460] Webserver started at http://127.0.236.66:35095/ using document root <none> and password file <none>
I20260806 15:05:24.576930 19115 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:24.576979 19115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:05:24.577097 19115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 15:05:24 dist-test-slave-tc2s krb5kdc[18046](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028724, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for HTTP/127.0.236.84@KRBTEST.COM
I20260806 15:05:24.597375 19115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/data/instance:
uuid: "ec1c240558c0409d80703f63f7e726ba"
format_stamp: "Formatted at 2026-08-06 15:05:24 on dist-test-slave-tc2s"
server_key: "7ca85e474cd3e36d4bcb9b88048b6ff2"
server_key_iv: "21da5e31caa5cd23ce6fd3a60c3afe9b"
server_key_version: "kuduclusterkey@0"
I20260806 15:05:24.597790 19115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/wal/instance:
uuid: "ec1c240558c0409d80703f63f7e726ba"
format_stamp: "Formatted at 2026-08-06 15:05:24 on dist-test-slave-tc2s"
server_key: "7ca85e474cd3e36d4bcb9b88048b6ff2"
server_key_iv: "21da5e31caa5cd23ce6fd3a60c3afe9b"
server_key_version: "kuduclusterkey@0"
I20260806 15:05:24.599531 19115 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:05:24.611128 19138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:24.611274 19115 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:05:24.611358 19115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "ec1c240558c0409d80703f63f7e726ba"
format_stamp: "Formatted at 2026-08-06 15:05:24 on dist-test-slave-tc2s"
server_key: "7ca85e474cd3e36d4bcb9b88048b6ff2"
server_key_iv: "21da5e31caa5cd23ce6fd3a60c3afe9b"
server_key_version: "kuduclusterkey@0"
I20260806 15:05:24.611441 19115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:24.623965 19115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:24.625671 19115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:24.625793 19115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:24.626030 19115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:05:24.626348 19115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:05:24.626389 19115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:24.626418 19115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:05:24.626438 19115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:24.633059 19115 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:42299
I20260806 15:05:24.633476 19115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
I20260806 15:05:24.634047 19329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:42299 every 8 connection(s)
I20260806 15:05:24.638803 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 19115
I20260806 15:05:24.638921 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/wal/instance
I20260806 15:05:24.639099 945 external_mini_cluster.cc:1428] Setting key 5682746d66f9c94761e1b1a22ea145d8
Aug 06 15:05:24 dist-test-slave-tc2s krb5kdc[18046](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028724, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:05:24.642995 19109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:24.634455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:48731 (local address 127.0.236.126:38613)
0806 15:05:24.634518 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:24.634520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:24.634639 (+ 119us) server_negotiation.cc:407] Connection header received
0806 15:05:24.635114 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:24.635116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:24.635129 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:24.635180 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:24.635704 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:24.636356 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:24.637522 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:24.637729 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:24.637784 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:24.639930 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:24.639931 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:24.639932 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:24.639939 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:24.641444 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:24.641952 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:24.641953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:24.641953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:24.642022 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:24.642298 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:24.642299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:24.642299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:24.642427 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:24.642467 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:24.642805 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 15:05:24.642905 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 15:05:24.643476 19332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:24.634182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:38613 (local address 127.0.236.66:48731)
0806 15:05:24.634529 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:24.634565 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:24.635025 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:24.635245 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:24.635249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:24.635431 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:24.635615 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:24.635620 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:24.636470 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:24.636471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:24.637437 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:24.637440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:24.637812 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:24.637813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:24.637963 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:24.638387 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:24.638395 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:24.639820 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:24.641531 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:24.641538 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:24.641540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:24.641808 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:24.642094 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:24.642095 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:24.642095 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:24.642230 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:24.642510 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:24.642517 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:24.642666 (+ 149us) client_negotiation.cc:769] Sending connection context
0806 15:05:24.643116 (+ 450us) client_negotiation.cc:240] Negotiation successful
0806 15:05:24.643267 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":74,"threads_started":1}
I20260806 15:05:24.644081 19330 heartbeater.cc:347] Connected to a master server at 127.0.236.126:38613
I20260806 15:05:24.644169 19330 heartbeater.cc:464] Registering TS with master...
I20260806 15:05:24.644333 19330 heartbeater.cc:511] Master 127.0.236.126:38613 requested a full tablet report, sending...
I20260806 15:05:24.644690 18785 ts_manager.cc:194] Registered new tserver with Master: ec1c240558c0409d80703f63f7e726ba (127.0.236.66:42299)
I20260806 15:05:24.645035 18785 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.66@KRBTEST.COM'} at 127.0.236.66:48731
WARNING: no policy specified for kudu/127.0.236.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
I20260806 15:05:24.687095 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.236.84:40643/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:0
--local_ip_for_outbound_sockets=127.0.236.67
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:38613
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:44515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:05:24.802821 19335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:24.803171 19335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:24.803257 19335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:24.804970 19335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:24.805028 19335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:24.805068 19335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:05:24.807271 19335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:44515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.0.236.84:40643/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:38613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:24.807498 19335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:24.807770 19335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:05:24.814145 19348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:24.814438 19344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:24.814450 19343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:24.815029 19335 server_base.cc:1034] running on GCE node
I20260806 15:05:24.815249 19335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:24.815516 19335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:24.816692 19335 hybrid_clock.cc:648] HybridClock initialized: now 1786028724816669 us; error 37 us; skew 500 ppm
Aug 06 15:05:24 dist-test-slave-tc2s krb5kdc[18046](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028724, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:24.820091 19335 init.cc:377] Logged in from keytab as kudu/127.0.236.67@KRBTEST.COM (short username kudu)
I20260806 15:05:24.820678 19335 webserver.cc:460] Webserver started at http://127.0.236.67:45515/ using document root <none> and password file <none>
I20260806 15:05:24.820959 19335 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:24.821040 19335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:05:24.821189 19335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 15:05:24 dist-test-slave-tc2s krb5kdc[18046](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028724, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for HTTP/127.0.236.84@KRBTEST.COM
I20260806 15:05:24.841216 19335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/data/instance:
uuid: "3b1b84dcce814ac7a31b71fe88cec6d0"
format_stamp: "Formatted at 2026-08-06 15:05:24 on dist-test-slave-tc2s"
server_key: "c336eca204e22a20e18ac3356f22b27c"
server_key_iv: "762ea815d73359833887b667f0062c6b"
server_key_version: "kuduclusterkey@0"
I20260806 15:05:24.841698 19335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/wal/instance:
uuid: "3b1b84dcce814ac7a31b71fe88cec6d0"
format_stamp: "Formatted at 2026-08-06 15:05:24 on dist-test-slave-tc2s"
server_key: "c336eca204e22a20e18ac3356f22b27c"
server_key_iv: "762ea815d73359833887b667f0062c6b"
server_key_version: "kuduclusterkey@0"
I20260806 15:05:24.843611 19335 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260806 15:05:24.867208 19360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:24.875280 19335 fs_manager.cc:725] Time spent opening block manager: real 0.018s user 0.000s sys 0.000s
I20260806 15:05:24.875392 19335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "3b1b84dcce814ac7a31b71fe88cec6d0"
format_stamp: "Formatted at 2026-08-06 15:05:24 on dist-test-slave-tc2s"
server_key: "c336eca204e22a20e18ac3356f22b27c"
server_key_iv: "762ea815d73359833887b667f0062c6b"
server_key_version: "kuduclusterkey@0"
I20260806 15:05:24.875492 19335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:24.885032 19335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:24.887226 19335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:24.887358 19335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:24.887583 19335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:05:24.887913 19335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:05:24.887953 19335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:24.887981 19335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:05:24.888008 19335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:24.895601 19335 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:46425
I20260806 15:05:24.895712 19548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:46425 every 8 connection(s)
I20260806 15:05:24.896073 19335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
Aug 06 15:05:24 dist-test-slave-tc2s krb5kdc[18046](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028724, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:05:24.903818 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 19335
I20260806 15:05:24.903944 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/wal/instance
I20260806 15:05:24.904124 945 external_mini_cluster.cc:1428] Setting key e91cc6882ec8000acba0e91f45089856
I20260806 15:05:24.906682 19109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:24.896853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:60309 (local address 127.0.236.126:38613)
0806 15:05:24.896894 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:24.896896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:24.897128 (+ 232us) server_negotiation.cc:407] Connection header received
0806 15:05:24.897815 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:24.897817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:24.897830 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:24.897870 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:24.898646 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:24.899385 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:24.900754 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:24.901040 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:24.901076 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:24.903425 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:24.903427 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:24.903427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:24.903435 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:24.905177 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:24.905635 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:24.905636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:24.905637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:24.905698 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:24.906011 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:24.906012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:24.906013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:24.906136 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:24.906173 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:24.906510 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 15:05:24.906588 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 15:05:24.907188 19551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:24.896828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:38613 (local address 127.0.236.67:60309)
0806 15:05:24.897003 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:24.897047 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:24.897693 (+ 646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:24.898076 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:24.898081 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:24.898292 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:24.898547 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:24.898552 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:24.899492 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:24.899493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:24.900658 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:24.900661 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:24.901168 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:24.901169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:24.901293 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:24.901770 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:24.901780 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:24.903323 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:24.905259 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:24.905266 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:24.905269 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:24.905565 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:24.905770 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:24.905771 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:24.905771 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:24.905941 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:24.906213 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:24.906218 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:24.906364 (+ 146us) client_negotiation.cc:769] Sending connection context
0806 15:05:24.906808 (+ 444us) client_negotiation.cc:240] Negotiation successful
0806 15:05:24.906971 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
I20260806 15:05:24.907655 19549 heartbeater.cc:347] Connected to a master server at 127.0.236.126:38613
I20260806 15:05:24.907761 19549 heartbeater.cc:464] Registering TS with master...
I20260806 15:05:24.907953 19549 heartbeater.cc:511] Master 127.0.236.126:38613 requested a full tablet report, sending...
I20260806 15:05:24.908329 18785 ts_manager.cc:194] Registered new tserver with Master: 3b1b84dcce814ac7a31b71fe88cec6d0 (127.0.236.67:46425)
I20260806 15:05:24.908785 18785 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.67@KRBTEST.COM'} at 127.0.236.67:60309
I20260806 15:05:24.916332 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:05:24.916953 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 18911
I20260806 15:05:24.943524 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 19115
I20260806 15:05:24.954615 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 19335
I20260806 15:05:24.962430 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 18739
I20260806 15:05:24.970957 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:38613
--webserver_interface=127.0.236.126
--webserver_port=34653
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:44515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:38613
--ranger_config_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.236.84:40643/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.236.84:40643/kms with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:05:25.049825 19553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:25.050103 19553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:25.050140 19553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:25.051715 19553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:05:25.051769 19553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:25.051788 19553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:25.051805 19553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:05:25.051828 19553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:05:25.053622 19553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:44515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.236.84:40643/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:38613
--ranger_config_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:38613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=34653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:25.053798 19553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:25.054006 19553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:05:25.058353 19560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:25.058389 19561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:25.058389 19563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:25.058504 19553 server_base.cc:1034] running on GCE node
I20260806 15:05:25.058729 19553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:25.058986 19553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:25.060113 19553 hybrid_clock.cc:648] HybridClock initialized: now 1786028725060103 us; error 26 us; skew 500 ppm
Aug 06 15:05:25 dist-test-slave-tc2s krb5kdc[18046](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028725, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:25.062441 19553 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:05:25.062827 19553 webserver.cc:460] Webserver started at http://127.0.236.126:34653/ using document root <none> and password file <none>
I20260806 15:05:25.063010 19553 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:25.063063 19553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 15:05:25 dist-test-slave-tc2s krb5kdc[18046](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028725, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for HTTP/127.0.236.84@KRBTEST.COM
I20260806 15:05:25.130735 19553 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:05:25.131412 19580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:25.131559 19553 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:05:25.131645 19553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "dbb6d46f3c74487aa3ff246669bfd950"
format_stamp: "Formatted at 2026-08-06 15:05:23 on dist-test-slave-tc2s"
server_key: "a8fb65e87c8ca89ecf3b3f17604aac99"
server_key_iv: "2cfa2accc02c1abee6ed7e6a6a62f779"
server_key_version: "kuduclusterkey@0"
I20260806 15:05:25.132026 19553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:25.152431 19553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:25.153767 19553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:25.153915 19553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:25.157691 19553 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:38613
I20260806 15:05:25.157745 19661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:38613 every 8 connection(s)
I20260806 15:05:25.158105 19553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:05:25.159248 19664 sys_catalog.cc:263] Verifying existing consensus state
I20260806 15:05:25.159876 19664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950: Bootstrap starting.
I20260806 15:05:25.166359 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 19553
I20260806 15:05:25.166714 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.236.84:40643/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:37921
--local_ip_for_outbound_sockets=127.0.236.65
--tserver_master_addrs=127.0.236.126:38613
--webserver_port=36943
--webserver_interface=127.0.236.65
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:44515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.236.84:40643/kms with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:05:25.167910 19664 log.cc:824] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950: Log is configured to *not* fsync() on all Append() calls
I20260806 15:05:25.169497 19664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 15:05:25.169736 19664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950: Bootstrap complete.
I20260806 15:05:25.171907 19664 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb6d46f3c74487aa3ff246669bfd950" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 38613 } }
I20260806 15:05:25.172138 19664 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbb6d46f3c74487aa3ff246669bfd950, State: Initialized, Role: FOLLOWER
I20260806 15:05:25.172289 19664 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb6d46f3c74487aa3ff246669bfd950" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 38613 } }
I20260806 15:05:25.172363 19664 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:05:25.172411 19664 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:05:25.172456 19664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [term 1 FOLLOWER]: Advancing to term 2
I20260806 15:05:25.173377 19664 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb6d46f3c74487aa3ff246669bfd950" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 38613 } }
I20260806 15:05:25.173471 19664 leader_election.cc:302] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbb6d46f3c74487aa3ff246669bfd950; no voters:
I20260806 15:05:25.173622 19664 leader_election.cc:288] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 15:05:25.173669 19670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 15:05:25.173774 19670 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [term 2 LEADER]: Becoming Leader. State: Replica: dbb6d46f3c74487aa3ff246669bfd950, State: Running, Role: LEADER
I20260806 15:05:25.173859 19670 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb6d46f3c74487aa3ff246669bfd950" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 38613 } }
I20260806 15:05:25.173872 19664 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:05:25.174189 19671 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "dbb6d46f3c74487aa3ff246669bfd950" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb6d46f3c74487aa3ff246669bfd950" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 38613 } } }
I20260806 15:05:25.174270 19671 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [sys.catalog]: This master's current role is: LEADER
I20260806 15:05:25.174497 19672 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbb6d46f3c74487aa3ff246669bfd950. Latest consensus state: current_term: 2 leader_uuid: "dbb6d46f3c74487aa3ff246669bfd950" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb6d46f3c74487aa3ff246669bfd950" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 38613 } } }
I20260806 15:05:25.174605 19672 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950 [sys.catalog]: This master's current role is: LEADER
I20260806 15:05:25.174739 19677 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:05:25.175239 19677 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:05:25.175371 19677 catalog_manager.cc:1259] Loaded cluster ID: 42b741347bb14033ad8b1be9794ef245
I20260806 15:05:25.175411 19677 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:05:25.175530 19664 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 15:05:25.178889 19677 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:05:25.179404 19677 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950: Loaded TSK: 0
I20260806 15:05:25.179657 19677 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 15:05:25.277835 19669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:25.278131 19669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:25.278162 19669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:25.279795 19669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:25.279850 19669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:25.279887 19669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:05:25.281495 19669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:44515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.236.84:40643/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:37921
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=36943
--tserver_master_addrs=127.0.236.126:38613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:25.281730 19669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:25.281980 19669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:05:25.287369 19723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:25.287359 19718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:25.287362 19721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:25.287798 19669 server_base.cc:1034] running on GCE node
I20260806 15:05:25.287999 19669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:25.288225 19669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:25.291453 19669 hybrid_clock.cc:648] HybridClock initialized: now 1786028725291440 us; error 25 us; skew 500 ppm
Aug 06 15:05:25 dist-test-slave-tc2s krb5kdc[18046](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028725, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:25.293931 19669 init.cc:377] Logged in from keytab as kudu/127.0.236.65@KRBTEST.COM (short username kudu)
I20260806 15:05:25.294387 19669 webserver.cc:460] Webserver started at http://127.0.236.65:36943/ using document root <none> and password file <none>
I20260806 15:05:25.294617 19669 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:25.294682 19669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 15:05:25 dist-test-slave-tc2s krb5kdc[18046](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028725, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for HTTP/127.0.236.84@KRBTEST.COM
I20260806 15:05:25.314347 19669 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.001s sys 0.000s
I20260806 15:05:25.316282 19733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:25.316496 19669 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 15:05:25.316597 19669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "14343bc961be4363822f94cce5a6fc5c"
format_stamp: "Formatted at 2026-08-06 15:05:24 on dist-test-slave-tc2s"
server_key: "3c6285e2461dca11ba3cdaffd68f2fc0"
server_key_iv: "e9ba80c602c297c261fe980502c8fef7"
server_key_version: "kuduclusterkey@0"
I20260806 15:05:25.317070 19669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:25.326812 19669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:25.328562 19669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:25.328692 19669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:25.328886 19669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:05:25.329216 19669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:05:25.329252 19669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:25.329279 19669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:05:25.329300 19669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:25.336130 19669 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:37921
I20260806 15:05:25.336212 19900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:37921 every 8 connection(s)
I20260806 15:05:25.336531 19669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:05:25.342983 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 19669
I20260806 15:05:25.343288 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.236.84:40643/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:42299
--local_ip_for_outbound_sockets=127.0.236.66
--tserver_master_addrs=127.0.236.126:38613
--webserver_port=35095
--webserver_interface=127.0.236.66
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:44515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.236.84:40643/kms with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 15:05:25 dist-test-slave-tc2s krb5kdc[18046](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028725, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:05:25.349812 19908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:25.337729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:36489 (local address 127.0.236.126:38613)
0806 15:05:25.339448 (+ 1719us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:25.339458 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:25.339501 (+ 43us) server_negotiation.cc:407] Connection header received
0806 15:05:25.339654 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:25.339660 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:25.339720 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:25.339861 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:25.340591 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:25.342040 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:25.343534 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:25.343957 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:25.344053 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:25.346258 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:25.346262 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:25.346263 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:25.346275 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:25.348032 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:25.348530 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:25.348533 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:25.348534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:25.348646 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:25.348919 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:25.348920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:25.348920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:25.349149 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:25.349221 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:25.349518 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 15:05:25.349621 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1661,"thread_start_us":86,"threads_started":1}
I20260806 15:05:25.349821 19907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:25.337315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:38613 (local address 127.0.236.65:36489)
0806 15:05:25.337501 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:25.337537 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:25.338174 (+ 637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:25.339926 (+ 1752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:25.339930 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:25.340161 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:25.340446 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:25.340453 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:25.342143 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:25.342144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:25.343291 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:25.343294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:25.344058 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:25.344059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:25.344175 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:25.344625 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:25.344633 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:25.346128 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:25.348125 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:25.348132 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:25.348134 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:25.348431 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:25.348723 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:25.348724 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:25.348724 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:25.348843 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:25.349214 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:25.349221 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:25.349363 (+ 142us) client_negotiation.cc:769] Sending connection context
0806 15:05:25.349451 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 15:05:25.349604 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":110,"threads_started":1}
I20260806 15:05:25.350507 19901 heartbeater.cc:347] Connected to a master server at 127.0.236.126:38613
I20260806 15:05:25.350595 19901 heartbeater.cc:464] Registering TS with master...
I20260806 15:05:25.350735 19901 heartbeater.cc:511] Master 127.0.236.126:38613 requested a full tablet report, sending...
W20260806 15:05:25.351500 19901 heartbeater.cc:644] Failed to heartbeat to 127.0.236.126:38613 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 15:05:25.351558 19901 heartbeater.cc:464] Registering TS with master...
I20260806 15:05:25.351619 19901 heartbeater.cc:511] Master 127.0.236.126:38613 requested a full tablet report, sending...
I20260806 15:05:25.351898 19901 heartbeater.cc:464] Registering TS with master...
I20260806 15:05:25.351966 19901 heartbeater.cc:511] Master 127.0.236.126:38613 requested a full tablet report, sending...
W20260806 15:05:25.352217 19901 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 15:05:25.464684 19911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:25.464944 19911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:25.464985 19911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:25.466627 19911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:25.466676 19911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:25.466710 19911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:05:25.468794 19911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:44515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.236.84:40643/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:42299
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=35095
--tserver_master_addrs=127.0.236.126:38613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:25.469017 19911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:25.469249 19911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:05:25.475075 19918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:25.475061 19922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:25.475051 19919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:25.475363 19911 server_base.cc:1034] running on GCE node
I20260806 15:05:25.475550 19911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:25.475806 19911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:25.479044 19911 hybrid_clock.cc:648] HybridClock initialized: now 1786028725479022 us; error 33 us; skew 500 ppm
Aug 06 15:05:25 dist-test-slave-tc2s krb5kdc[18046](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028725, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:25.481586 19911 init.cc:377] Logged in from keytab as kudu/127.0.236.66@KRBTEST.COM (short username kudu)
I20260806 15:05:25.482064 19911 webserver.cc:460] Webserver started at http://127.0.236.66:35095/ using document root <none> and password file <none>
I20260806 15:05:25.482291 19911 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:25.482364 19911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 15:05:25 dist-test-slave-tc2s krb5kdc[18046](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028725, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for HTTP/127.0.236.84@KRBTEST.COM
I20260806 15:05:25.499253 19911 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20260806 15:05:25.500147 19934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:25.500288 19911 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:05:25.500381 19911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "ec1c240558c0409d80703f63f7e726ba"
format_stamp: "Formatted at 2026-08-06 15:05:24 on dist-test-slave-tc2s"
server_key: "7ca85e474cd3e36d4bcb9b88048b6ff2"
server_key_iv: "21da5e31caa5cd23ce6fd3a60c3afe9b"
server_key_version: "kuduclusterkey@0"
I20260806 15:05:25.500738 19911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:25.523790 19911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:25.525746 19911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:25.525959 19911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:25.526206 19911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:05:25.526589 19911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:05:25.526628 19911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:25.526659 19911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:05:25.526680 19911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:25.533386 19911 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:42299
I20260806 15:05:25.533488 20108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:42299 every 8 connection(s)
I20260806 15:05:25.533785 19911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
Aug 06 15:05:25 dist-test-slave-tc2s krb5kdc[18046](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028725, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:05:25.540417 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 19911
I20260806 15:05:25.540696 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.236.84:40643/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:46425
--local_ip_for_outbound_sockets=127.0.236.67
--tserver_master_addrs=127.0.236.126:38613
--webserver_port=45515
--webserver_interface=127.0.236.67
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:44515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.236.84:40643/kms with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:05:25.544070 19908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:25.534703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:55715 (local address 127.0.236.126:38613)
0806 15:05:25.534758 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:25.534760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:25.535010 (+ 250us) server_negotiation.cc:407] Connection header received
0806 15:05:25.535532 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:25.535533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:25.535549 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:25.535605 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:25.536338 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:25.536883 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:25.537916 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:25.538164 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:25.538236 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:25.540534 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:25.540535 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:25.540536 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:25.540543 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:25.542425 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:25.542931 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:25.542933 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:25.542933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:25.543001 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:25.543298 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:25.543300 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:25.543300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:25.543446 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:25.543514 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:25.543879 (+ 365us) server_negotiation.cc:299] Negotiation successful
0806 15:05:25.543975 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 15:05:25.544366 20112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:25.534730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:38613 (local address 127.0.236.66:55715)
0806 15:05:25.534890 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:25.534925 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:25.535414 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:25.535717 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:25.535722 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:25.535957 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:25.536248 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:25.536254 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:25.536970 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:25.536971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:25.537804 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:25.537806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:25.538284 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:25.538286 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:25.538442 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:25.538917 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:25.538927 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:25.540406 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:25.542518 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:25.542525 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:25.542527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:25.542831 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:25.543081 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:25.543081 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:25.543082 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:25.543218 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:25.543550 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:25.543556 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:25.543718 (+ 162us) client_negotiation.cc:769] Sending connection context
0806 15:05:25.543831 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 15:05:25.543982 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
I20260806 15:05:25.544762 20110 heartbeater.cc:347] Connected to a master server at 127.0.236.126:38613
I20260806 15:05:25.544852 20110 heartbeater.cc:464] Registering TS with master...
I20260806 15:05:25.545015 20110 heartbeater.cc:511] Master 127.0.236.126:38613 requested a full tablet report, sending...
W20260806 15:05:25.545714 20110 heartbeater.cc:644] Failed to heartbeat to 127.0.236.126:38613 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 15:05:25.545779 20110 heartbeater.cc:464] Registering TS with master...
I20260806 15:05:25.545851 20110 heartbeater.cc:511] Master 127.0.236.126:38613 requested a full tablet report, sending...
I20260806 15:05:25.546170 20110 heartbeater.cc:464] Registering TS with master...
I20260806 15:05:25.546234 20110 heartbeater.cc:511] Master 127.0.236.126:38613 requested a full tablet report, sending...
W20260806 15:05:25.546497 20110 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 15:05:25.641489 20113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:25.641736 20113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:25.641767 20113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:25.643447 20113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:25.643504 20113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:25.643540 20113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:05:25.645684 20113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:44515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.236.84:40643/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:46425
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=45515
--tserver_master_addrs=127.0.236.126:38613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:25.645938 20113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:25.646169 20113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:05:25.652202 20128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:25.652218 20124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:25.652973 20113 server_base.cc:1034] running on GCE node
W20260806 15:05:25.652371 20132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:25.654150 20113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:25.654425 20113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:25.655596 20113 hybrid_clock.cc:648] HybridClock initialized: now 1786028725655569 us; error 43 us; skew 500 ppm
Aug 06 15:05:25 dist-test-slave-tc2s krb5kdc[18046](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028725, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:25.658864 20113 init.cc:377] Logged in from keytab as kudu/127.0.236.67@KRBTEST.COM (short username kudu)
I20260806 15:05:25.659401 20113 webserver.cc:460] Webserver started at http://127.0.236.67:45515/ using document root <none> and password file <none>
I20260806 15:05:25.659714 20113 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:25.659834 20113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 15:05:25 dist-test-slave-tc2s krb5kdc[18046](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028725, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for HTTP/127.0.236.84@KRBTEST.COM
I20260806 15:05:25.694347 20113 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.000s
I20260806 15:05:25.695442 20145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:25.695648 20113 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:05:25.695757 20113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "3b1b84dcce814ac7a31b71fe88cec6d0"
format_stamp: "Formatted at 2026-08-06 15:05:24 on dist-test-slave-tc2s"
server_key: "c336eca204e22a20e18ac3356f22b27c"
server_key_iv: "762ea815d73359833887b667f0062c6b"
server_key_version: "kuduclusterkey@0"
I20260806 15:05:25.696223 20113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:25.711535 20113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:25.713238 20113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:25.713366 20113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:25.713582 20113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:05:25.713876 20113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:05:25.713944 20113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:25.713979 20113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:05:25.714001 20113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:25.720829 20113 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:46425
I20260806 15:05:25.720927 20323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:46425 every 8 connection(s)
I20260806 15:05:25.721285 20113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
Aug 06 15:05:25 dist-test-slave-tc2s krb5kdc[18046](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028725, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:05:25.729871 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 20113
I20260806 15:05:25.733636 19908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:25.721997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:52857 (local address 127.0.236.126:38613)
0806 15:05:25.722051 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:25.722053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:25.722331 (+ 278us) server_negotiation.cc:407] Connection header received
0806 15:05:25.722806 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:25.722807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:25.722819 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:25.722864 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:25.724960 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:25.725698 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:25.727215 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:25.727553 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:25.727599 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:25.730402 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:25.730404 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:25.730404 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:25.730413 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:25.732009 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:25.732537 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:25.732539 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:25.732539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:25.732631 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:25.732956 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:25.732957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:25.732958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:25.733075 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:25.733127 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:25.733479 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 15:05:25.733549 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 15:05:25.734103 20327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:25.722001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:38613 (local address 127.0.236.67:52857)
0806 15:05:25.722200 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:25.722241 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:25.722698 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:25.724419 (+ 1721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:25.724424 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:25.724602 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:25.724841 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:25.724847 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:25.725817 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:25.725817 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:25.727105 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:25.727108 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:25.727739 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:25.727740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:25.727886 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:25.728376 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:25.728386 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:25.730279 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:25.732107 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:25.732114 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:25.732116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:25.732421 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:25.732717 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:25.732718 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:25.732719 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:25.732865 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:25.733145 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:25.733151 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:25.733316 (+ 165us) client_negotiation.cc:769] Sending connection context
0806 15:05:25.733444 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 15:05:25.733604 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":92,"threads_started":1}
I20260806 15:05:25.734619 20324 heartbeater.cc:347] Connected to a master server at 127.0.236.126:38613
I20260806 15:05:25.734721 20324 heartbeater.cc:464] Registering TS with master...
I20260806 15:05:25.734915 20324 heartbeater.cc:511] Master 127.0.236.126:38613 requested a full tablet report, sending...
W20260806 15:05:25.735535 20324 heartbeater.cc:644] Failed to heartbeat to 127.0.236.126:38613 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 15:05:25.735607 20324 heartbeater.cc:464] Registering TS with master...
I20260806 15:05:25.735667 20324 heartbeater.cc:511] Master 127.0.236.126:38613 requested a full tablet report, sending...
I20260806 15:05:25.735982 20324 heartbeater.cc:464] Registering TS with master...
I20260806 15:05:25.736068 20324 heartbeater.cc:511] Master 127.0.236.126:38613 requested a full tablet report, sending...
W20260806 15:05:25.736361 20324 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 15:05:25.741526 19908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:25.730316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36320 (local address 127.0.236.126:38613)
0806 15:05:25.733709 (+ 3393us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:25.733711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:25.733718 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:05:25.733735 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:25.733735 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:25.733744 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:25.733801 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:25.736722 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:25.737160 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:25.737935 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:25.738138 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:25.738195 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:25.739278 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:25.739280 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:25.739280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:25.739286 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:25.740547 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:25.740846 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:25.740847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:25.740847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:25.740898 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:25.741064 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:25.741065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:25.741065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:25.741167 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:25.741326 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:25.741361 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 15:05:25.741430 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3379,"thread_start_us":64,"threads_started":1}
Aug 06 15:05:26 dist-test-slave-tc2s krb5kdc[18046](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786028726, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:26.352377 19901 heartbeater.cc:464] Registering TS with master...
I20260806 15:05:26.352517 19901 heartbeater.cc:511] Master 127.0.236.126:38613 requested a full tablet report, sending...
I20260806 15:05:26.352833 19609 ts_manager.cc:194] Registered new tserver with Master: 14343bc961be4363822f94cce5a6fc5c (127.0.236.65:37921)
I20260806 15:05:26.353819 19609 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.65@KRBTEST.COM'} at 127.0.236.65:36489
I20260806 15:05:26.546633 20110 heartbeater.cc:464] Registering TS with master...
I20260806 15:05:26.546777 20110 heartbeater.cc:511] Master 127.0.236.126:38613 requested a full tablet report, sending...
I20260806 15:05:26.547030 19609 ts_manager.cc:194] Registered new tserver with Master: ec1c240558c0409d80703f63f7e726ba (127.0.236.66:42299)
I20260806 15:05:26.547539 19609 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.66@KRBTEST.COM'} at 127.0.236.66:55715
Aug 06 15:05:26 dist-test-slave-tc2s krb5kdc[18046](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028726, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for HTTP/127.0.236.84@KRBTEST.COM
I20260806 15:05:26.736545 20324 heartbeater.cc:464] Registering TS with master...
I20260806 15:05:26.736707 20324 heartbeater.cc:511] Master 127.0.236.126:38613 requested a full tablet report, sending...
I20260806 15:05:26.737017 19609 ts_manager.cc:194] Registered new tserver with Master: 3b1b84dcce814ac7a31b71fe88cec6d0 (127.0.236.67:46425)
I20260806 15:05:26.737532 19609 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.67@KRBTEST.COM'} at 127.0.236.67:52857
I20260806 15:05:26.743857 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:05:26.752555 20360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:26.744847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36334 (local address 127.0.236.126:38613)
0806 15:05:26.745133 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:26.745135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:26.745143 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:05:26.745164 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:26.745164 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:26.745175 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:26.745244 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:26.745957 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:26.746739 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:26.747809 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:26.748140 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:26.748207 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:26.749686 (+ 1479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:26.749688 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:26.749688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:26.749696 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:26.751325 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:26.751728 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:26.751729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:26.751729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:26.751795 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:26.752012 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:26.752013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:26.752013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:26.752139 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:26.752228 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:26.752376 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 15:05:26.752460 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":70,"threads_started":1}
I20260806 15:05:26.755324 19609 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36334:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:05:26.757685 19609 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:05:26.766135 20367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:26.760621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:37921 (local address 127.0.0.1:47128)
0806 15:05:26.761066 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:26.761366 (+ 300us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:26.761414 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:26.762445 (+ 1031us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:26.762446 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:26.762448 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:05:26.762498 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:26.762500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:26.763679 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:26.763680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:26.764873 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:26.764876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:26.765828 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:26.765829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:26.765937 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:26.765953 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 15:05:26.766015 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 15:05:26.766063 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":81,"threads_started":1}
I20260806 15:05:26.766659 20371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:26.760884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47128 (local address 127.0.236.65:37921)
0806 15:05:26.762235 (+ 1351us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:26.762239 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:26.762247 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:05:26.762279 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:26.762281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:26.762296 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:26.762373 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:05:26.762564 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:26.763586 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:26.764966 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:26.765660 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:26.765735 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:26.765793 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:26.766542 (+ 749us) server_negotiation.cc:299] Negotiation successful
0806 15:05:26.766586 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1322,"thread_start_us":73,"threads_started":1}
I20260806 15:05:26.766726 20370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:26.760853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:46425 (local address 127.0.0.1:39292)
0806 15:05:26.762065 (+ 1212us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:26.762074 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:26.762131 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:26.762411 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:26.762414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:26.762416 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:05:26.762482 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:26.762484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:26.764085 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:26.764086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:26.765169 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:26.765172 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:26.766494 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:26.766495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:26.766557 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:26.766561 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:05:26.766612 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 15:05:26.766650 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1198,"thread_start_us":65,"threads_started":1}
I20260806 15:05:26.766958 20372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:26.760939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39292 (local address 127.0.236.67:46425)
0806 15:05:26.762011 (+ 1072us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:26.762015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:26.762146 (+ 131us) server_negotiation.cc:407] Connection header received
0806 15:05:26.762189 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:26.762192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:26.762208 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:26.762295 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:05:26.762564 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:26.763985 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:26.765537 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:26.766305 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:26.766393 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:26.766453 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:26.766829 (+ 376us) server_negotiation.cc:299] Negotiation successful
0806 15:05:26.766873 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1037,"thread_start_us":45,"threads_started":1}
I20260806 15:05:26.767614 19801 tablet_service.cc:1467] Processing CreateTablet for tablet a3fbd5509b7e4637824d69ffdc386344 (DEFAULT_TABLE table=test-table [id=d38ce3bd73514dbb8de506a4becb9c74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:05:26.767614 20222 tablet_service.cc:1467] Processing CreateTablet for tablet a3fbd5509b7e4637824d69ffdc386344 (DEFAULT_TABLE table=test-table [id=d38ce3bd73514dbb8de506a4becb9c74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:05:26.767930 19801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3fbd5509b7e4637824d69ffdc386344. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:05:26.767935 20222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3fbd5509b7e4637824d69ffdc386344. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:05:26.770588 20374 tablet_bootstrap.cc:492] T a3fbd5509b7e4637824d69ffdc386344 P 14343bc961be4363822f94cce5a6fc5c: Bootstrap starting.
I20260806 15:05:26.771287 20374 tablet_bootstrap.cc:654] T a3fbd5509b7e4637824d69ffdc386344 P 14343bc961be4363822f94cce5a6fc5c: Neither blocks nor log segments found. Creating new log.
I20260806 15:05:26.771878 20374 log.cc:824] T a3fbd5509b7e4637824d69ffdc386344 P 14343bc961be4363822f94cce5a6fc5c: Log is configured to *not* fsync() on all Append() calls
I20260806 15:05:26.772730 20374 tablet_bootstrap.cc:492] T a3fbd5509b7e4637824d69ffdc386344 P 14343bc961be4363822f94cce5a6fc5c: No bootstrap required, opened a new log
I20260806 15:05:26.772807 20374 ts_tablet_manager.cc:1397] T a3fbd5509b7e4637824d69ffdc386344 P 14343bc961be4363822f94cce5a6fc5c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:05:26.775319 20374 raft_consensus.cc:348] T a3fbd5509b7e4637824d69ffdc386344 P 14343bc961be4363822f94cce5a6fc5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14343bc961be4363822f94cce5a6fc5c" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 37921 } } peers { permanent_uuid: "ec1c240558c0409d80703f63f7e726ba" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42299 } } peers { permanent_uuid: "3b1b84dcce814ac7a31b71fe88cec6d0" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46425 } }
I20260806 15:05:26.775486 20374 raft_consensus.cc:374] T a3fbd5509b7e4637824d69ffdc386344 P 14343bc961be4363822f94cce5a6fc5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:05:26.775537 20374 raft_consensus.cc:729] T a3fbd5509b7e4637824d69ffdc386344 P 14343bc961be4363822f94cce5a6fc5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14343bc961be4363822f94cce5a6fc5c, State: Initialized, Role: FOLLOWER
I20260806 15:05:26.775980 20374 consensus_queue.cc:260] T a3fbd5509b7e4637824d69ffdc386344 P 14343bc961be4363822f94cce5a6fc5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14343bc961be4363822f94cce5a6fc5c" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 37921 } } peers { permanent_uuid: "ec1c240558c0409d80703f63f7e726ba" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42299 } } peers { permanent_uuid: "3b1b84dcce814ac7a31b71fe88cec6d0" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46425 } }
I20260806 15:05:26.776242 20374 ts_tablet_manager.cc:1428] T a3fbd5509b7e4637824d69ffdc386344 P 14343bc961be4363822f94cce5a6fc5c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 15:05:26.776348 19901 heartbeater.cc:503] Master 127.0.236.126:38613 was elected leader, sending a full tablet report...
I20260806 15:05:26.778046 20375 tablet_bootstrap.cc:492] T a3fbd5509b7e4637824d69ffdc386344 P 3b1b84dcce814ac7a31b71fe88cec6d0: Bootstrap starting.
I20260806 15:05:26.778142 20369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:26.760623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:42299 (local address 127.0.0.1:38662)
0806 15:05:26.761180 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:26.761221 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:26.761282 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:26.774192 (+ 12910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:26.774192 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:26.774195 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:05:26.774233 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:26.774235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:26.775780 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:26.775781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:26.776832 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:26.776835 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:26.777863 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:26.777864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:26.777971 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:26.777975 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:05:26.778030 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 15:05:26.778068 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":84,"threads_started":1}
I20260806 15:05:26.778441 20373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:26.761045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38662 (local address 127.0.236.66:42299)
0806 15:05:26.774023 (+ 12978us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:26.774027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:26.774040 (+ 13us) server_negotiation.cc:407] Connection header received
0806 15:05:26.774073 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:26.774075 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:26.774090 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:26.774292 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:05:26.774419 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:26.775695 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:26.777033 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:26.777758 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:26.777841 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:26.777925 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:26.778321 (+ 396us) server_negotiation.cc:299] Negotiation successful
0806 15:05:26.778366 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12940,"thread_start_us":54,"threads_started":1}
I20260806 15:05:26.778956 20375 tablet_bootstrap.cc:654] T a3fbd5509b7e4637824d69ffdc386344 P 3b1b84dcce814ac7a31b71fe88cec6d0: Neither blocks nor log segments found. Creating new log.
I20260806 15:05:26.779076 20005 tablet_service.cc:1467] Processing CreateTablet for tablet a3fbd5509b7e4637824d69ffdc386344 (DEFAULT_TABLE table=test-table [id=d38ce3bd73514dbb8de506a4becb9c74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:05:26.779419 20005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3fbd5509b7e4637824d69ffdc386344. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:05:26.779546 20375 log.cc:824] T a3fbd5509b7e4637824d69ffdc386344 P 3b1b84dcce814ac7a31b71fe88cec6d0: Log is configured to *not* fsync() on all Append() calls
I20260806 15:05:26.782925 20385 tablet_bootstrap.cc:492] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba: Bootstrap starting.
I20260806 15:05:26.783638 20385 tablet_bootstrap.cc:654] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba: Neither blocks nor log segments found. Creating new log.
I20260806 15:05:26.783939 20385 log.cc:824] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba: Log is configured to *not* fsync() on all Append() calls
I20260806 15:05:26.784701 20385 tablet_bootstrap.cc:492] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba: No bootstrap required, opened a new log
I20260806 15:05:26.784767 20385 ts_tablet_manager.cc:1397] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:05:26.786542 20375 tablet_bootstrap.cc:492] T a3fbd5509b7e4637824d69ffdc386344 P 3b1b84dcce814ac7a31b71fe88cec6d0: No bootstrap required, opened a new log
I20260806 15:05:26.786612 20375 ts_tablet_manager.cc:1397] T a3fbd5509b7e4637824d69ffdc386344 P 3b1b84dcce814ac7a31b71fe88cec6d0: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260806 15:05:26.787448 20385 raft_consensus.cc:348] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14343bc961be4363822f94cce5a6fc5c" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 37921 } } peers { permanent_uuid: "ec1c240558c0409d80703f63f7e726ba" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42299 } } peers { permanent_uuid: "3b1b84dcce814ac7a31b71fe88cec6d0" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46425 } }
I20260806 15:05:26.787590 20385 raft_consensus.cc:374] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:05:26.787621 20385 raft_consensus.cc:729] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec1c240558c0409d80703f63f7e726ba, State: Initialized, Role: FOLLOWER
I20260806 15:05:26.787743 20385 consensus_queue.cc:260] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14343bc961be4363822f94cce5a6fc5c" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 37921 } } peers { permanent_uuid: "ec1c240558c0409d80703f63f7e726ba" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42299 } } peers { permanent_uuid: "3b1b84dcce814ac7a31b71fe88cec6d0" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46425 } }
I20260806 15:05:26.787977 20385 ts_tablet_manager.cc:1428] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 15:05:26.789057 20375 raft_consensus.cc:348] T a3fbd5509b7e4637824d69ffdc386344 P 3b1b84dcce814ac7a31b71fe88cec6d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14343bc961be4363822f94cce5a6fc5c" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 37921 } } peers { permanent_uuid: "ec1c240558c0409d80703f63f7e726ba" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42299 } } peers { permanent_uuid: "3b1b84dcce814ac7a31b71fe88cec6d0" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46425 } }
I20260806 15:05:26.789202 20375 raft_consensus.cc:374] T a3fbd5509b7e4637824d69ffdc386344 P 3b1b84dcce814ac7a31b71fe88cec6d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:05:26.789240 20375 raft_consensus.cc:729] T a3fbd5509b7e4637824d69ffdc386344 P 3b1b84dcce814ac7a31b71fe88cec6d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b1b84dcce814ac7a31b71fe88cec6d0, State: Initialized, Role: FOLLOWER
I20260806 15:05:26.789350 20375 consensus_queue.cc:260] T a3fbd5509b7e4637824d69ffdc386344 P 3b1b84dcce814ac7a31b71fe88cec6d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14343bc961be4363822f94cce5a6fc5c" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 37921 } } peers { permanent_uuid: "ec1c240558c0409d80703f63f7e726ba" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42299 } } peers { permanent_uuid: "3b1b84dcce814ac7a31b71fe88cec6d0" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46425 } }
I20260806 15:05:26.789444 20110 heartbeater.cc:503] Master 127.0.236.126:38613 was elected leader, sending a full tablet report...
I20260806 15:05:26.789594 20375 ts_tablet_manager.cc:1428] T a3fbd5509b7e4637824d69ffdc386344 P 3b1b84dcce814ac7a31b71fe88cec6d0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 15:05:26.789808 20324 heartbeater.cc:503] Master 127.0.236.126:38613 was elected leader, sending a full tablet report...
W20260806 15:05:26.838929 19903 tablet.cc:2369] T a3fbd5509b7e4637824d69ffdc386344 P 14343bc961be4363822f94cce5a6fc5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 15:05:26 dist-test-slave-tc2s krb5kdc[18046](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028726, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for HTTP/127.0.236.84@KRBTEST.COM
W20260806 15:05:26.972530 20325 tablet.cc:2369] T a3fbd5509b7e4637824d69ffdc386344 P 3b1b84dcce814ac7a31b71fe88cec6d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 15:05:27.035158 20111 tablet.cc:2369] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:05:27.100754 20388 raft_consensus.cc:482] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:05:27.100906 20388 raft_consensus.cc:504] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14343bc961be4363822f94cce5a6fc5c" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 37921 } } peers { permanent_uuid: "ec1c240558c0409d80703f63f7e726ba" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42299 } } peers { permanent_uuid: "3b1b84dcce814ac7a31b71fe88cec6d0" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46425 } }
I20260806 15:05:27.101333 20388 leader_election.cc:288] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14343bc961be4363822f94cce5a6fc5c (127.0.236.65:37921), 3b1b84dcce814ac7a31b71fe88cec6d0 (127.0.236.67:46425)
I20260806 15:05:27.106101 20371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:27.102186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:60339 (local address 127.0.236.65:37921)
0806 15:05:27.102243 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:27.102246 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:27.102405 (+ 159us) server_negotiation.cc:407] Connection header received
0806 15:05:27.102519 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:27.102520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:27.102530 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:27.102560 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:05:27.102772 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:27.103546 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:27.105133 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:27.105676 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:27.105724 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:27.105733 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:27.105985 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 15:05:27.106027 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 15:05:27.106554 20393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:27.101638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:37921 (local address 127.0.236.66:60339)
0806 15:05:27.102338 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:27.102349 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:27.102444 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:27.102627 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:27.102628 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:27.102629 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:05:27.102705 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:27.102707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:27.103754 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:27.103755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:27.105029 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:27.105033 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:27.105809 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:27.105810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:27.105880 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:27.105884 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:05:27.106398 (+ 514us) client_negotiation.cc:240] Negotiation successful
0806 15:05:27.106454 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"thread_start_us":82,"threads_started":1}
I20260806 15:05:27.106985 19832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3fbd5509b7e4637824d69ffdc386344" candidate_uuid: "ec1c240558c0409d80703f63f7e726ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14343bc961be4363822f94cce5a6fc5c" is_pre_election: true
I20260806 15:05:27.107136 19832 raft_consensus.cc:2454] T a3fbd5509b7e4637824d69ffdc386344 P 14343bc961be4363822f94cce5a6fc5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec1c240558c0409d80703f63f7e726ba in term 0.
I20260806 15:05:27.107446 19937 leader_election.cc:302] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14343bc961be4363822f94cce5a6fc5c, ec1c240558c0409d80703f63f7e726ba; no voters:
I20260806 15:05:27.107568 20388 raft_consensus.cc:2781] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:05:27.107606 20388 raft_consensus.cc:482] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:05:27.107627 20388 raft_consensus.cc:3037] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:05:27.108538 20388 raft_consensus.cc:504] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14343bc961be4363822f94cce5a6fc5c" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 37921 } } peers { permanent_uuid: "ec1c240558c0409d80703f63f7e726ba" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42299 } } peers { permanent_uuid: "3b1b84dcce814ac7a31b71fe88cec6d0" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46425 } }
I20260806 15:05:27.108662 20388 leader_election.cc:288] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba [CANDIDATE]: Term 1 election: Requested vote from peers 14343bc961be4363822f94cce5a6fc5c (127.0.236.65:37921), 3b1b84dcce814ac7a31b71fe88cec6d0 (127.0.236.67:46425)
I20260806 15:05:27.108980 19832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3fbd5509b7e4637824d69ffdc386344" candidate_uuid: "ec1c240558c0409d80703f63f7e726ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14343bc961be4363822f94cce5a6fc5c"
I20260806 15:05:27.109057 19832 raft_consensus.cc:3037] T a3fbd5509b7e4637824d69ffdc386344 P 14343bc961be4363822f94cce5a6fc5c [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:05:27.110467 19832 raft_consensus.cc:2454] T a3fbd5509b7e4637824d69ffdc386344 P 14343bc961be4363822f94cce5a6fc5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec1c240558c0409d80703f63f7e726ba in term 1.
I20260806 15:05:27.110677 19937 leader_election.cc:302] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14343bc961be4363822f94cce5a6fc5c, ec1c240558c0409d80703f63f7e726ba; no voters:
I20260806 15:05:27.110788 20388 raft_consensus.cc:2781] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:05:27.110924 20388 raft_consensus.cc:686] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba [term 1 LEADER]: Becoming Leader. State: Replica: ec1c240558c0409d80703f63f7e726ba, State: Running, Role: LEADER
I20260806 15:05:27.111006 20388 consensus_queue.cc:237] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14343bc961be4363822f94cce5a6fc5c" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 37921 } } peers { permanent_uuid: "ec1c240558c0409d80703f63f7e726ba" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42299 } } peers { permanent_uuid: "3b1b84dcce814ac7a31b71fe88cec6d0" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46425 } }
I20260806 15:05:27.111788 19609 catalog_manager.cc:5495] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba reported cstate change: term changed from 0 to 1, leader changed from <none> to ec1c240558c0409d80703f63f7e726ba (127.0.236.66). New cstate: current_term: 1 leader_uuid: "ec1c240558c0409d80703f63f7e726ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14343bc961be4363822f94cce5a6fc5c" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 37921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec1c240558c0409d80703f63f7e726ba" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b1b84dcce814ac7a31b71fe88cec6d0" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46425 } health_report { overall_health: UNKNOWN } } }
I20260806 15:05:27.115485 20372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:27.102259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:47379 (local address 127.0.236.67:46425)
0806 15:05:27.102304 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:27.102306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:27.112360 (+ 10054us) server_negotiation.cc:407] Connection header received
0806 15:05:27.112478 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:27.112479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:27.112490 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:27.112517 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:05:27.112684 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:27.113322 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:27.114578 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:27.115124 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:27.115173 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:27.115181 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:27.115369 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 15:05:27.115404 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 15:05:27.115816 20393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:27.101833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:46425 (local address 127.0.236.66:47379)
0806 15:05:27.112295 (+ 10462us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:27.112306 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:27.112388 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:27.112581 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:27.112583 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:27.112584 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:05:27.112614 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:27.112615 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:27.113432 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:27.113433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:27.114475 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:27.114478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:27.115236 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:27.115237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:27.115297 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:27.115300 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:05:27.115711 (+ 411us) client_negotiation.cc:240] Negotiation successful
0806 15:05:27.115752 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10446,"thread_start_us":93,"threads_started":1}
I20260806 15:05:27.116292 20253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3fbd5509b7e4637824d69ffdc386344" candidate_uuid: "ec1c240558c0409d80703f63f7e726ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b1b84dcce814ac7a31b71fe88cec6d0" is_pre_election: true
I20260806 15:05:27.116446 20253 raft_consensus.cc:2454] T a3fbd5509b7e4637824d69ffdc386344 P 3b1b84dcce814ac7a31b71fe88cec6d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec1c240558c0409d80703f63f7e726ba in term 0.
I20260806 15:05:27.116566 20249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3fbd5509b7e4637824d69ffdc386344" candidate_uuid: "ec1c240558c0409d80703f63f7e726ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b1b84dcce814ac7a31b71fe88cec6d0"
I20260806 15:05:27.116626 20249 raft_consensus.cc:3037] T a3fbd5509b7e4637824d69ffdc386344 P 3b1b84dcce814ac7a31b71fe88cec6d0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:05:27.117460 20249 raft_consensus.cc:2454] T a3fbd5509b7e4637824d69ffdc386344 P 3b1b84dcce814ac7a31b71fe88cec6d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec1c240558c0409d80703f63f7e726ba in term 1.
I20260806 15:05:27.201784 20373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:27.199263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38674 (local address 127.0.236.66:42299)
0806 15:05:27.199328 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:27.199330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:27.199359 (+ 29us) server_negotiation.cc:407] Connection header received
0806 15:05:27.199408 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:27.199409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:27.199435 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:27.199543 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:05:27.199698 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:27.200214 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:27.201064 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:27.201262 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:27.201312 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:27.201476 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:05:27.201580 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:05:27.201660 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:27.201683 (+ 23us) server_negotiation.cc:299] Negotiation successful
0806 15:05:27.201705 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 15:05:27.203663 19832 raft_consensus.cc:1261] T a3fbd5509b7e4637824d69ffdc386344 P 14343bc961be4363822f94cce5a6fc5c [term 1 FOLLOWER]: Refusing update from remote peer ec1c240558c0409d80703f63f7e726ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:05:27.203995 20249 raft_consensus.cc:1261] T a3fbd5509b7e4637824d69ffdc386344 P 3b1b84dcce814ac7a31b71fe88cec6d0 [term 1 FOLLOWER]: Refusing update from remote peer ec1c240558c0409d80703f63f7e726ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:05:27.204319 20388 consensus_queue.cc:1035] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "14343bc961be4363822f94cce5a6fc5c" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 37921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:05:27.204443 20397 consensus_queue.cc:1035] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b1b84dcce814ac7a31b71fe88cec6d0" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:05:27.208257 20411 mvcc.cc:205] Tried to move back new op lower bound from 7315573666623336448 to 7315573666247184384. Current Snapshot: MvccSnapshot[applied={T|T < 7315573666623336448}]
I20260806 15:05:27.209205 20413 mvcc.cc:205] Tried to move back new op lower bound from 7315573666623336448 to 7315573666247184384. Current Snapshot: MvccSnapshot[applied={T|T < 7315573666623336448}]
I20260806 15:05:27.211560 19608 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:05:27.211655 19608 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:05:27.212421 19608 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dbb6d46f3c74487aa3ff246669bfd950: Sending DeleteTablet for 3 replicas of tablet a3fbd5509b7e4637824d69ffdc386344
I20260806 15:05:27.213024 19801 tablet_service.cc:1514] Processing DeleteTablet for tablet a3fbd5509b7e4637824d69ffdc386344 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:05:27 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:47128
I20260806 15:05:27.213240 20005 tablet_service.cc:1514] Processing DeleteTablet for tablet a3fbd5509b7e4637824d69ffdc386344 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:05:27 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:38662
I20260806 15:05:27.213444 20222 tablet_service.cc:1514] Processing DeleteTablet for tablet a3fbd5509b7e4637824d69ffdc386344 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:05:27 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:39292
I20260806 15:05:27.213928 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 19669
I20260806 15:05:27.220666 20424 tablet_replica.cc:330] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba: stopping tablet replica
I20260806 15:05:27.220777 20424 raft_consensus.cc:2229] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:05:27.220896 20424 raft_consensus.cc:2258] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:05:27.221040 20425 tablet_replica.cc:330] T a3fbd5509b7e4637824d69ffdc386344 P 3b1b84dcce814ac7a31b71fe88cec6d0: stopping tablet replica
I20260806 15:05:27.221125 20425 raft_consensus.cc:2229] T a3fbd5509b7e4637824d69ffdc386344 P 3b1b84dcce814ac7a31b71fe88cec6d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:05:27.221217 20425 raft_consensus.cc:2258] T a3fbd5509b7e4637824d69ffdc386344 P 3b1b84dcce814ac7a31b71fe88cec6d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:05:27.226322 20425 ts_tablet_manager.cc:1905] T a3fbd5509b7e4637824d69ffdc386344 P 3b1b84dcce814ac7a31b71fe88cec6d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:05:27.226322 20424 ts_tablet_manager.cc:1905] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:05:27.228089 20424 ts_tablet_manager.cc:1918] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:05:27.228168 20424 log.cc:1196] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-1/wal/wals/a3fbd5509b7e4637824d69ffdc386344
I20260806 15:05:27.228425 20424 ts_tablet_manager.cc:1939] T a3fbd5509b7e4637824d69ffdc386344 P ec1c240558c0409d80703f63f7e726ba: Deleting consensus metadata
I20260806 15:05:27.228806 19588 catalog_manager.cc:4841] TS ec1c240558c0409d80703f63f7e726ba (127.0.236.66:42299): tablet a3fbd5509b7e4637824d69ffdc386344 (table test-table [id=d38ce3bd73514dbb8de506a4becb9c74]) successfully deleted
W20260806 15:05:27.229035 19584 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.236.65:37921 (error 108)
I20260806 15:05:27.229882 20369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:27.229750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:37921 (local address 127.0.0.1:47142)
0806 15:05:27.229800 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:27.229846 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.65:37921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 15:05:27.230024 19584 catalog_manager.cc:4584] TS 14343bc961be4363822f94cce5a6fc5c (127.0.236.65:37921): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a3fbd5509b7e4637824d69ffdc386344: Network error: Client connection negotiation failed: client connection to 127.0.236.65:37921: connect: Connection refused (error 111)
I20260806 15:05:27.230207 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 19911
I20260806 15:05:27.234015 20425 ts_tablet_manager.cc:1918] T a3fbd5509b7e4637824d69ffdc386344 P 3b1b84dcce814ac7a31b71fe88cec6d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:05:27.234706 20425 log.cc:1196] T a3fbd5509b7e4637824d69ffdc386344 P 3b1b84dcce814ac7a31b71fe88cec6d0: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/a3fbd5509b7e4637824d69ffdc386344
I20260806 15:05:27.235124 20425 ts_tablet_manager.cc:1939] T a3fbd5509b7e4637824d69ffdc386344 P 3b1b84dcce814ac7a31b71fe88cec6d0: Deleting consensus metadata
I20260806 15:05:27.259990 19587 catalog_manager.cc:4841] TS 3b1b84dcce814ac7a31b71fe88cec6d0 (127.0.236.67:46425): tablet a3fbd5509b7e4637824d69ffdc386344 (table test-table [id=d38ce3bd73514dbb8de506a4becb9c74]) successfully deleted
I20260806 15:05:27.264599 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 20113
I20260806 15:05:27.272490 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 19553
I20260806 15:05:27.286649 945 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 15:05:27.632475 945 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 15:05:27.632565 945 mini_ranger.cc:66] Stopping Ranger...
I20260806 15:05:27.975703 945 mini_ranger.cc:68] Stopped Ranger
2026-08-06 15:05:27.975 UTC [18099] LOG: received smart shutdown request
2026-08-06 15:05:27.979 UTC [18099] LOG: background worker "logical replication launcher" (PID 18116) exited with exit code 1
2026-08-06 15:05:27.979 UTC [18105] LOG: shutting down
2026-08-06 15:05:28.096 UTC [18099] LOG: database system is shut down
2026-08-06T15:05:28Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (44712 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:05:28 dist-test-slave-tc2s krb5kdc[20441](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:05:28 dist-test-slave-tc2s krb5kdc[20441](info): set up 2 sockets
Aug 06 15:05:28 dist-test-slave-tc2s krb5kdc[20441](info): commencing operation
krb5kdc: starting...
W20260806 15:05:29.300021 945 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:05:29 dist-test-slave-tc2s krb5kdc[20441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:05:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:05:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:05:29.414887 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:33083
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:36447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:33083,127.0.236.125:42395,127.0.236.124:35597
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:05:29.495325 20460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:29.495522 20460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:29.495543 20460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:29.496596 20460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:05:29.496627 20460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:29.496640 20460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:29.496651 20460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:05:29.496668 20460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:05:29.498116 20460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:36447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:33083,127.0.236.125:42395,127.0.236.124:35597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:33083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:29.498301 20460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:29.498469 20460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:05:29.503027 20466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:29.503014 20472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:29.503011 20469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:29.503048 20460 server_base.cc:1034] running on GCE node
I20260806 15:05:29.503454 20460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:29.503718 20460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:29.504868 20460 hybrid_clock.cc:648] HybridClock initialized: now 1786028729504851 us; error 33 us; skew 500 ppm
Aug 06 15:05:29 dist-test-slave-tc2s krb5kdc[20441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028729, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:29.509967 20460 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:05:29.510352 20460 webserver.cc:460] Webserver started at http://127.0.236.126:37483/ using document root <none> and password file <none>
I20260806 15:05:29.510540 20460 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:29.510586 20460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:05:29.510764 20460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:05:29.511786 20460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "2bb5ff98bbd640cf9dc4c48628ba98ef"
format_stamp: "Formatted at 2026-08-06 15:05:29 on dist-test-slave-tc2s"
server_key: "e7dd873e311629e50e89a63fc713af18"
server_key_iv: "59d2ba32dd54adc51a03f3de239e90d7"
server_key_version: "encryptionkey@0"
I20260806 15:05:29.512100 20460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "2bb5ff98bbd640cf9dc4c48628ba98ef"
format_stamp: "Formatted at 2026-08-06 15:05:29 on dist-test-slave-tc2s"
server_key: "e7dd873e311629e50e89a63fc713af18"
server_key_iv: "59d2ba32dd54adc51a03f3de239e90d7"
server_key_version: "encryptionkey@0"
I20260806 15:05:29.513283 20460 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:05:29.513952 20481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:29.514101 20460 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:05:29.514182 20460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "2bb5ff98bbd640cf9dc4c48628ba98ef"
format_stamp: "Formatted at 2026-08-06 15:05:29 on dist-test-slave-tc2s"
server_key: "e7dd873e311629e50e89a63fc713af18"
server_key_iv: "59d2ba32dd54adc51a03f3de239e90d7"
server_key_version: "encryptionkey@0"
I20260806 15:05:29.514247 20460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:29.527133 20460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:29.529023 20460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:29.529157 20460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:29.532977 20460 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:33083
I20260806 15:05:29.533010 20560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:33083 every 8 connection(s)
I20260806 15:05:29.533305 20460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:05:29.534134 20561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:05:29.535468 20561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.126" port: 33083 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:05:29.540023 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 20460
I20260806 15:05:29.540119 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:05:29.540285 945 external_mini_cluster.cc:1428] Setting key cdf7ad141b3c03cf24a38c15ed398532
Aug 06 15:05:29 dist-test-slave-tc2s krb5kdc[20441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028729, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:05:29.543828 20563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.536026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:33083 (local address 127.0.0.1:52746)
0806 15:05:29.536170 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:29.536207 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:29.536635 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:29.536838 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:29.536839 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:29.537057 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:29.537201 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:29.537204 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.538157 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.538158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:29.538811 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:29.538812 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.539217 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.539218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:29.539268 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:29.539680 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:29.539688 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:29.540978 (+ 1290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:29.542518 (+ 1540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:29.542525 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:29.542527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:29.542767 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:29.542962 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:29.542963 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:29.542963 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:29.543055 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:29.543447 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:29.543450 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:29.543502 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 15:05:29.543567 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 15:05:29.543683 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":65,"threads_started":1}
I20260806 15:05:29.543855 20566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.536056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52746 (local address 127.0.236.126:33083)
0806 15:05:29.536207 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:29.536215 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:29.536282 (+ 67us) server_negotiation.cc:407] Connection header received
0806 15:05:29.536739 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:29.536741 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:29.536756 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:29.536824 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:29.537263 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.538081 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.538872 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.539164 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.539229 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:29.541055 (+ 1826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:29.541064 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:29.541065 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:29.541071 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:29.542443 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.542838 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.542840 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.542840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.542908 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.543116 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.543117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.543118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.543362 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:29.543460 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:29.543609 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 15:05:29.543757 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260806 15:05:29.544477 20561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.125" port: 42395 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:05:29.544798 20563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.544691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:42395 (local address 127.0.0.1:56406)
0806 15:05:29.544735 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:29.544779 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.125:42395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260806 15:05:29.544929 20490 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.236.125:42395: connect: Connection refused (error 111)
I20260806 15:05:29.545310 20563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.545230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:42395 (local address 127.0.0.1:56408)
0806 15:05:29.545271 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:29.545294 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.125:42395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 15:05:29.545418 20561 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.236.125:42395: Network error: Client connection negotiation failed: client connection to 127.0.236.125:42395: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.0.236.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.125.keytab.
Entry for principal kudu/127.0.236.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.125.keytab.
I20260806 15:05:29.562196 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.125:42395
--webserver_interface=127.0.236.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.125.keytab
--principal=kudu/127.0.236.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:36447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:33083,127.0.236.125:42395,127.0.236.124:35597
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:05:29.594630 20561 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.236.125" port: 42395 } attempt: 1
I20260806 15:05:29.595038 20563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.594918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:42395 (local address 127.0.0.1:56412)
0806 15:05:29.594972 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:29.595009 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.125:42395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 15:05:29.595403 20563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.595295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:42395 (local address 127.0.0.1:56422)
0806 15:05:29.595337 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:29.595362 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.125:42395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 15:05:29.595489 20561 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.236.125:42395: Network error: Client connection negotiation failed: client connection to 127.0.236.125:42395: connect: Connection refused (error 111)
W20260806 15:05:29.639856 20572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:29.640040 20572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:29.640069 20572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:29.641155 20572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:05:29.641193 20572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:29.641213 20572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:29.641229 20572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:05:29.641245 20572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:05:29.642712 20572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:36447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:33083,127.0.236.125:42395,127.0.236.124:35597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.125.keytab
--principal=kudu/127.0.236.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.125:42395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.236.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:29.642936 20572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:29.643131 20572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:05:29.647601 20582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:29.647612 20585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:29.647619 20587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:29.647804 20572 server_base.cc:1034] running on GCE node
I20260806 15:05:29.648029 20572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:29.648310 20572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:29.649443 20572 hybrid_clock.cc:648] HybridClock initialized: now 1786028729649430 us; error 28 us; skew 500 ppm
Aug 06 15:05:29 dist-test-slave-tc2s krb5kdc[20441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028729, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:29.651707 20572 init.cc:377] Logged in from keytab as kudu/127.0.236.125@KRBTEST.COM (short username kudu)
I20260806 15:05:29.652118 20572 webserver.cc:460] Webserver started at http://127.0.236.125:46353/ using document root <none> and password file <none>
I20260806 15:05:29.652319 20572 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:29.652365 20572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:05:29.652487 20572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:05:29.653409 20572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-1/data/instance:
uuid: "2d2cca91c70c4ea8a7ebc4b90c098967"
format_stamp: "Formatted at 2026-08-06 15:05:29 on dist-test-slave-tc2s"
server_key: "9657efda70f0bc492c4d7206b945c3ef"
server_key_iv: "35eee1f09f66eb8a5ee013a1d03ce9d5"
server_key_version: "encryptionkey@0"
I20260806 15:05:29.653822 20572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-1/wal/instance:
uuid: "2d2cca91c70c4ea8a7ebc4b90c098967"
format_stamp: "Formatted at 2026-08-06 15:05:29 on dist-test-slave-tc2s"
server_key: "9657efda70f0bc492c4d7206b945c3ef"
server_key_iv: "35eee1f09f66eb8a5ee013a1d03ce9d5"
server_key_version: "encryptionkey@0"
I20260806 15:05:29.655049 20572 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:05:29.655665 20600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:29.655819 20572 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:05:29.655905 20572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-1/wal
uuid: "2d2cca91c70c4ea8a7ebc4b90c098967"
format_stamp: "Formatted at 2026-08-06 15:05:29 on dist-test-slave-tc2s"
server_key: "9657efda70f0bc492c4d7206b945c3ef"
server_key_iv: "35eee1f09f66eb8a5ee013a1d03ce9d5"
server_key_version: "encryptionkey@0"
I20260806 15:05:29.655970 20572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:29.663609 20561 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.236.125" port: 42395 } attempt: 2
I20260806 15:05:29.663925 20563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.663828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:42395 (local address 127.0.0.1:56426)
0806 15:05:29.663874 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:29.663900 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.125:42395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 15:05:29.664201 20563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.664131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:42395 (local address 127.0.0.1:56434)
0806 15:05:29.664162 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:29.664185 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.125:42395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260806 15:05:29.664271 20561 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.236.125:42395: Network error: Client connection negotiation failed: client connection to 127.0.236.125:42395: connect: Connection refused (error 111)
I20260806 15:05:29.666985 20572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:29.668478 20572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:29.668591 20572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:29.672170 20572 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.125:42395
I20260806 15:05:29.672200 20679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.125:42395 every 8 connection(s)
I20260806 15:05:29.672473 20572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-1/data/info.pb
I20260806 15:05:29.673231 20680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:05:29.674618 20680 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.126" port: 33083 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:05:29.676162 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 20572
I20260806 15:05:29.676277 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-1/wal/instance
I20260806 15:05:29.676465 945 external_mini_cluster.cc:1428] Setting key bc7dc5f05ada96630667582c936fe9c5
Aug 06 15:05:29 dist-test-slave-tc2s krb5kdc[20441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028729, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.125@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:05:29.683490 20566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.675182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52754 (local address 127.0.236.126:33083)
0806 15:05:29.675226 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:29.675227 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:29.675391 (+ 164us) server_negotiation.cc:407] Connection header received
0806 15:05:29.675793 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:29.675794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:29.675804 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:29.675852 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:29.676566 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.677047 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.678316 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.678611 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.678674 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:29.680799 (+ 2125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:29.680801 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:29.680801 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:29.680808 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:29.682268 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.682626 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.682627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.682627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.682689 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.682911 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.682912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.682912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.683047 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:29.683107 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:29.683318 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 15:05:29.683400 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 15:05:29.683543 20682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.675151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:33083 (local address 127.0.0.1:52754)
0806 15:05:29.675292 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:29.675329 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:29.675725 (+ 396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:29.675918 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:29.675922 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:29.676213 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:29.676485 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:29.676490 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.677137 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.677138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:29.678241 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:29.678244 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.678683 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.678684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:29.678775 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:29.679193 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:29.679201 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:29.680713 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:29.682344 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:29.682350 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:29.682352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:29.682560 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:29.682755 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:29.682756 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:29.682756 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:29.682853 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:29.683106 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:29.683109 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:29.683210 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 15:05:29.683279 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 15:05:29.683389 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":57,"threads_started":1}
I20260806 15:05:29.684032 20680 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.125" port: 42395 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.0.236.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.124@KRBTEST.COM" created.
Aug 06 15:05:29 dist-test-slave-tc2s krb5kdc[20441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028729, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.125@KRBTEST.COM for kudu/127.0.236.125@KRBTEST.COM
I20260806 15:05:29.690845 20682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.684253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:42395 (local address 127.0.0.1:56438)
0806 15:05:29.684305 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:29.684316 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:29.684360 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:29.684564 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:29.684565 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:29.684697 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:29.684748 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:29.684749 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.685571 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.685572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:29.686101 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:29.686103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.686503 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.686504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:29.686550 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:29.686824 (+ 274us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:29.686827 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:29.688205 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:29.689993 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:29.689994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:29.689994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:29.690141 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:29.690342 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:29.690342 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:29.690343 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:29.690385 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:29.690653 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:29.690654 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:29.690685 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 15:05:29.690729 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 15:05:29.690788 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 15:05:29.691053 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.684297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56438 (local address 127.0.236.125:42395)
0806 15:05:29.684425 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:29.684429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:29.684454 (+ 25us) server_negotiation.cc:407] Connection header received
0806 15:05:29.684475 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:29.684476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:29.684486 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:29.684544 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:29.684813 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.685499 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.686165 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.686452 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.686500 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:29.688281 (+ 1781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:29.688289 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:29.688289 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:29.688294 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:29.689885 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.690215 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.690218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.690218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.690282 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.690433 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.690434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.690434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.690597 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:29.690677 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:29.690775 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 15:05:29.690858 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":68,"threads_started":1}
I20260806 15:05:29.691581 20680 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:05:29.691843 20682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.691745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:35597 (local address 127.0.0.1:53590)
0806 15:05:29.691782 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:29.691823 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:35597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 15:05:29.691958 20602 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.236.124:35597: connect: Connection refused (error 111)
I20260806 15:05:29.692284 20682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.692205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:35597 (local address 127.0.0.1:53594)
0806 15:05:29.692240 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:29.692262 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:35597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 15:05:29.692368 20680 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.236.124:35597: Network error: Client connection negotiation failed: client connection to 127.0.236.124:35597: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.124.keytab.
Entry for principal kudu/127.0.236.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.124.keytab.
I20260806 15:05:29.700222 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.124:35597
--webserver_interface=127.0.236.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.124.keytab
--principal=kudu/127.0.236.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:36447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:33083,127.0.236.125:42395,127.0.236.124:35597
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:05:29.741587 20680 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } attempt: 1
I20260806 15:05:29.742056 20682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.741942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:35597 (local address 127.0.0.1:53604)
0806 15:05:29.741994 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:29.742025 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:35597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 15:05:29.742342 20682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.742266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:35597 (local address 127.0.0.1:53608)
0806 15:05:29.742301 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:29.742325 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:35597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 15:05:29.742429 20680 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.236.124:35597: Network error: Client connection negotiation failed: client connection to 127.0.236.124:35597: connect: Connection refused (error 111)
I20260806 15:05:29.755398 20561 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.236.125" port: 42395 } attempt: 3
Aug 06 15:05:29 dist-test-slave-tc2s krb5kdc[20441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028729, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for kudu/127.0.236.125@KRBTEST.COM
I20260806 15:05:29.761853 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.755748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56440 (local address 127.0.236.125:42395)
0806 15:05:29.755804 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:29.755806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:29.755815 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:05:29.755840 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:29.755841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:29.755853 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:29.755899 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:29.756189 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.756685 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.757331 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.757551 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.757606 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:29.759664 (+ 2058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:29.759666 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:29.759667 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:29.759674 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:29.760849 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.761152 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.761153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.761154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.761208 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.761370 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.761371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.761371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.761487 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:29.761542 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:29.761676 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 15:05:29.761758 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 15:05:29.761855 20563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.755677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:42395 (local address 127.0.0.1:56440)
0806 15:05:29.755733 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:29.755743 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:29.755788 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:29.755909 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:29.755910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:29.756063 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:29.756124 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:29.756126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.756758 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.756759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:29.757261 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:29.757263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.757612 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.757613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:29.757685 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:29.758180 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:29.758184 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:29.759595 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:29.760931 (+ 1336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:29.760932 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:29.760933 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:29.761093 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:29.761261 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:29.761262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:29.761262 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:29.761322 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:29.761541 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:29.761541 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:29.761590 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 15:05:29.761649 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 15:05:29.761756 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 15:05:29.762326 20561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:05:29.762657 20563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.762563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:35597 (local address 127.0.0.1:53620)
0806 15:05:29.762605 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:29.762631 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:35597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 15:05:29.763000 20563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.762922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:35597 (local address 127.0.0.1:53632)
0806 15:05:29.762961 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:29.762983 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:35597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 15:05:29.763079 20561 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.236.124:35597: Network error: Client connection negotiation failed: client connection to 127.0.236.124:35597: connect: Connection refused (error 111)
W20260806 15:05:29.778738 20689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:29.778911 20689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:29.778931 20689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:29.780016 20689 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:05:29.780057 20689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:29.780071 20689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:29.780089 20689 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:05:29.780100 20689 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:05:29.781544 20689 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:36447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:33083,127.0.236.125:42395,127.0.236.124:35597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.124.keytab
--principal=kudu/127.0.236.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.124:35597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.236.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:29.781723 20689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:29.781878 20689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:05:29.786190 20699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:29.786239 20706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:29.786223 20700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:29.786486 20689 server_base.cc:1034] running on GCE node
I20260806 15:05:29.786727 20689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:29.787003 20689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:29.788143 20689 hybrid_clock.cc:648] HybridClock initialized: now 1786028729788127 us; error 25 us; skew 500 ppm
Aug 06 15:05:29 dist-test-slave-tc2s krb5kdc[20441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028729, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:29.790329 20689 init.cc:377] Logged in from keytab as kudu/127.0.236.124@KRBTEST.COM (short username kudu)
I20260806 15:05:29.790707 20689 webserver.cc:460] Webserver started at http://127.0.236.124:36705/ using document root <none> and password file <none>
I20260806 15:05:29.790879 20689 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:29.790916 20689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:05:29.791003 20689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:05:29.791899 20689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-2/data/instance:
uuid: "d2f132ee0bf94d92901757bc89cde263"
format_stamp: "Formatted at 2026-08-06 15:05:29 on dist-test-slave-tc2s"
server_key: "5e92756930c8deb52e0e0dc6609cb396"
server_key_iv: "3aac6b59f6992bae7e5639d7fb59e345"
server_key_version: "encryptionkey@0"
I20260806 15:05:29.792177 20689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-2/wal/instance:
uuid: "d2f132ee0bf94d92901757bc89cde263"
format_stamp: "Formatted at 2026-08-06 15:05:29 on dist-test-slave-tc2s"
server_key: "5e92756930c8deb52e0e0dc6609cb396"
server_key_iv: "3aac6b59f6992bae7e5639d7fb59e345"
server_key_version: "encryptionkey@0"
I20260806 15:05:29.793260 20689 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:05:29.793841 20714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:29.794000 20689 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:29.794088 20689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-2/wal
uuid: "d2f132ee0bf94d92901757bc89cde263"
format_stamp: "Formatted at 2026-08-06 15:05:29 on dist-test-slave-tc2s"
server_key: "5e92756930c8deb52e0e0dc6609cb396"
server_key_iv: "3aac6b59f6992bae7e5639d7fb59e345"
server_key_version: "encryptionkey@0"
I20260806 15:05:29.794147 20689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:29.794209 20561 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } attempt: 1
I20260806 15:05:29.794516 20563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.794419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:35597 (local address 127.0.0.1:53648)
0806 15:05:29.794465 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:29.794492 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:35597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 15:05:29.794812 20563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.794758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:35597 (local address 127.0.0.1:53662)
0806 15:05:29.794779 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:29.794799 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:35597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260806 15:05:29.794890 20561 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.236.124:35597: Network error: Client connection negotiation failed: client connection to 127.0.236.124:35597: connect: Connection refused (error 111)
I20260806 15:05:29.810554 20680 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } attempt: 2
I20260806 15:05:29.810864 20682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.810765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:35597 (local address 127.0.0.1:53668)
0806 15:05:29.810813 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:29.810838 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:35597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 15:05:29.811146 20682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.811073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:35597 (local address 127.0.0.1:53684)
0806 15:05:29.811105 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:29.811128 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:35597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260806 15:05:29.811221 20680 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.236.124:35597: Network error: Client connection negotiation failed: client connection to 127.0.236.124:35597: connect: Connection refused (error 111)
I20260806 15:05:29.815109 20689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:29.816737 20689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:29.816857 20689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:29.820479 20689 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.124:35597
I20260806 15:05:29.820509 20794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.124:35597 every 8 connection(s)
I20260806 15:05:29.820791 20689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-2/data/info.pb
I20260806 15:05:29.821573 20795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:05:29.823001 20795 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.126" port: 33083 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:05:29.824385 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 20689
I20260806 15:05:29.824479 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028580791223-945-0/minicluster-data/master-2/wal/instance
I20260806 15:05:29.824628 945 external_mini_cluster.cc:1428] Setting key 74b85f431ae2f49f042427ec4ab699bc
Aug 06 15:05:29 dist-test-slave-tc2s krb5kdc[20441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028729, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.124@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
Aug 06 15:05:29 dist-test-slave-tc2s krb5kdc[20441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.124@KRBTEST.COM
I20260806 15:05:29.831317 20566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.823571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52762 (local address 127.0.236.126:33083)
0806 15:05:29.823617 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:29.823618 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:29.823793 (+ 175us) server_negotiation.cc:407] Connection header received
0806 15:05:29.824226 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:29.824227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:29.824261 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:29.824310 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:29.824942 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.825547 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.826489 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.826684 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.826740 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:29.828836 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:29.828837 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:29.828838 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:29.828844 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:29.830059 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.830480 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.830481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.830481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.830535 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.830773 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.830773 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.830774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.830875 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:29.830924 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:29.831202 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 15:05:29.831261 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 15:05:29.831564 20797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.823559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:33083 (local address 127.0.0.1:52762)
0806 15:05:29.823691 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:29.823722 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:29.824137 (+ 415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:29.824371 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:29.824374 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:29.824643 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:29.824846 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:29.824850 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.825621 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.825623 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:29.826412 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:29.826414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.826751 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.826752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:29.826867 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:29.827280 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:29.827287 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:29.828763 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:29.830127 (+ 1364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:29.830133 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:29.830135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:29.830413 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:29.830592 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:29.830592 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:29.830593 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:29.830713 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:29.830936 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:29.830940 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:29.831083 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 15:05:29.831191 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 15:05:29.831355 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":57,"threads_started":1}
I20260806 15:05:29.832011 20795 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.125" port: 42395 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:05:29.832765 20801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.824949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53696 (local address 127.0.236.124:35597)
0806 15:05:29.825110 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:29.825114 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:29.825139 (+ 25us) server_negotiation.cc:407] Connection header received
0806 15:05:29.825161 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:29.825163 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:29.825182 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:29.825258 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:29.825784 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.826537 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.827298 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.827601 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.827657 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:29.829918 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:29.829925 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:29.829926 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:29.829932 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:29.831541 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.831900 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.831902 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.831902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.831956 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.832130 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.832131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.832132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.832287 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:29.832411 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:29.832599 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 15:05:29.832669 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":103,"threads_started":1}
Aug 06 15:05:29 dist-test-slave-tc2s krb5kdc[20441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028729, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.124@KRBTEST.COM for kudu/127.0.236.125@KRBTEST.COM
Aug 06 15:05:29 dist-test-slave-tc2s krb5kdc[20441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:05:29.838004 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.832288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56446 (local address 127.0.236.125:42395)
0806 15:05:29.832329 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:29.832330 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:29.832338 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:05:29.832359 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:29.832360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:29.832370 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:29.832432 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:29.832646 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.833246 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.833861 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.834151 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.834206 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:29.835686 (+ 1480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:29.835687 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:29.835687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:29.835693 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:29.837053 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.837337 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.837338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.837339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.837394 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.837568 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.837569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.837569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.837659 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:29.837700 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:29.837836 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 15:05:29.837920 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:05:29.838050 20797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.832212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:42395 (local address 127.0.0.1:56446)
0806 15:05:29.832250 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:29.832261 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:29.832296 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:29.832433 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:29.832434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:29.832535 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:29.832578 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:29.832579 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.833322 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.833322 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:29.833801 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:29.833803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.834212 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.834213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:29.834280 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:29.834603 (+ 323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:29.834607 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:29.835613 (+ 1006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:29.837126 (+ 1513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:29.837127 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:29.837127 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:29.837274 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:29.837452 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:29.837452 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:29.837453 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:29.837511 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:29.837708 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:29.837709 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:29.837750 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 15:05:29.837820 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 15:05:29.837919 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 15:05:29.838403 20795 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:05:29.840817 20566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.833434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52768 (local address 127.0.236.126:33083)
0806 15:05:29.833468 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:29.833469 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:29.833946 (+ 477us) server_negotiation.cc:407] Connection header received
0806 15:05:29.833997 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:29.833998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:29.834006 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:29.834069 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:29.834435 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.834900 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.836053 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.836230 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.836279 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:29.838435 (+ 2156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:29.838436 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:29.838436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:29.838443 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:29.839824 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.840147 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.840147 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.840147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.840189 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.840344 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.840345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.840345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.840459 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:29.840627 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:29.840662 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 15:05:29.840732 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Aug 06 15:05:29 dist-test-slave-tc2s krb5kdc[20441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028729, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.124@KRBTEST.COM for kudu/127.0.236.124@KRBTEST.COM
I20260806 15:05:29.844736 20797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.838590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:35597 (local address 127.0.0.1:53710)
0806 15:05:29.838634 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:29.838644 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:29.838678 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:29.838783 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:29.838784 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:29.838903 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:29.838929 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:29.838931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.839717 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.839718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:29.840342 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:29.840345 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.840808 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.840809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:29.840848 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:29.841167 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:29.841172 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:29.842327 (+ 1155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:29.843742 (+ 1415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:29.843743 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:29.843744 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:29.843937 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:29.844113 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:29.844114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:29.844115 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:29.844174 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:29.844413 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:29.844414 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:29.844458 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 15:05:29.844532 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 15:05:29.844620 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 15:05:29.844744 20801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.838644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53710 (local address 127.0.236.124:35597)
0806 15:05:29.838673 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:29.838675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:29.838683 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:05:29.838721 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:29.838722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:29.838730 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:29.838781 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:29.838992 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.839646 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.840408 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.840674 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.840736 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:29.842396 (+ 1660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:29.842397 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:29.842397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:29.842404 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:29.843667 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.843994 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.843995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.843995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.844052 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.844231 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.844231 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.844232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.844348 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:29.844419 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:29.844550 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 15:05:29.844662 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":14}
Aug 06 15:05:29 dist-test-slave-tc2s krb5kdc[20441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.125@KRBTEST.COM
I20260806 15:05:29.846203 20795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263: Bootstrap starting.
I20260806 15:05:29.846958 20795 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263: Neither blocks nor log segments found. Creating new log.
I20260806 15:05:29.847208 20795 log.cc:824] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263: Log is configured to *not* fsync() on all Append() calls
I20260806 15:05:29.847836 20795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263: No bootstrap required, opened a new log
I20260806 15:05:29.848169 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.841555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56454 (local address 127.0.236.125:42395)
0806 15:05:29.841593 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:29.841594 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:29.841638 (+ 44us) server_negotiation.cc:407] Connection header received
0806 15:05:29.841679 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:29.841680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:29.841688 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:29.841745 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:29.842185 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.842712 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.843398 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.843681 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.843739 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:29.846061 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:29.846062 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:29.846062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:29.846068 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:29.847244 (+ 1176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.847535 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.847536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.847537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.847582 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.847739 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.847740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.847740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.847830 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:29.847989 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:29.848024 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 15:05:29.848088 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:05:29.849426 20795 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bb5ff98bbd640cf9dc4c48628ba98ef" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 33083 } } peers { permanent_uuid: "2d2cca91c70c4ea8a7ebc4b90c098967" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 42395 } } peers { permanent_uuid: "d2f132ee0bf94d92901757bc89cde263" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } }
I20260806 15:05:29.849565 20795 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:05:29.849612 20795 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2f132ee0bf94d92901757bc89cde263, State: Initialized, Role: FOLLOWER
I20260806 15:05:29.849651 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:05:29.849721 20795 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bb5ff98bbd640cf9dc4c48628ba98ef" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 33083 } } peers { permanent_uuid: "2d2cca91c70c4ea8a7ebc4b90c098967" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 42395 } } peers { permanent_uuid: "d2f132ee0bf94d92901757bc89cde263" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } }
I20260806 15:05:29.849952 20805 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bb5ff98bbd640cf9dc4c48628ba98ef" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 33083 } } peers { permanent_uuid: "2d2cca91c70c4ea8a7ebc4b90c098967" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 42395 } } peers { permanent_uuid: "d2f132ee0bf94d92901757bc89cde263" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } } }
I20260806 15:05:29.850020 20805 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:05:29.850049 20795 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 15:05:29.851720 20833 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:05:29.851776 20833 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:05:29.852005 20689 master_runner.cc:186] Error getting master registration for 127.0.236.126:33083: OK, instance_id { permanent_uuid: "2bb5ff98bbd640cf9dc4c48628ba98ef" instance_seqno: 1786028729531949 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 15:05:29.854661 20689 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 15:05:29.858035 20566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.850998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52774 (local address 127.0.236.126:33083)
0806 15:05:29.851032 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:29.851034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:29.851141 (+ 107us) server_negotiation.cc:407] Connection header received
0806 15:05:29.851322 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:29.851323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:29.851333 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:29.851361 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:29.852168 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.852826 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.853790 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.854073 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.854114 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:29.855437 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:29.855438 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:29.855439 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:29.855445 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:29.856677 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.856998 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.856999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.856999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.857053 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.857212 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.857213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.857213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.857310 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:29.857817 (+ 507us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:29.857855 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 15:05:29.857955 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:05:29.858597 20801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.851012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53722 (local address 127.0.236.124:35597)
0806 15:05:29.851054 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:29.851056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:29.851292 (+ 236us) server_negotiation.cc:407] Connection header received
0806 15:05:29.851378 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:29.851379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:29.851387 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:29.851412 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:29.852287 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.852957 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.853957 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.854241 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.854279 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:29.855883 (+ 1604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:29.855885 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:29.855885 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:29.855891 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:29.857345 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.857736 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.857737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.857737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.857801 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.858010 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.858011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.858012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.858157 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:29.858241 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:29.858446 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 15:05:29.858522 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 15:05:29.861455 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.850880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56468 (local address 127.0.236.125:42395)
0806 15:05:29.851184 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:29.851186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:29.852360 (+ 1174us) server_negotiation.cc:407] Connection header received
0806 15:05:29.852442 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:29.852443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:29.852452 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:29.852477 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:29.853105 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.853781 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.854702 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.854964 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.854998 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:29.859343 (+ 4345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:29.859344 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:29.859345 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:29.859350 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:29.860509 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.860851 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.860851 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.860852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.860890 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.861063 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.861064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.861064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.861143 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:29.861201 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:29.861350 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 15:05:29.861398 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287}
I20260806 15:05:29.870026 20561 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } attempt: 2
Aug 06 15:05:29 dist-test-slave-tc2s krb5kdc[20441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028729, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for kudu/127.0.236.124@KRBTEST.COM
I20260806 15:05:29.875998 20801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.870318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53726 (local address 127.0.236.124:35597)
0806 15:05:29.870368 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:29.870369 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:29.870377 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:05:29.870395 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:29.870395 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:29.870403 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:29.870459 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:29.870661 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.871351 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.871915 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.872241 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.872304 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:29.873755 (+ 1451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:29.873757 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:29.873757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:29.873763 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:29.874942 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.875223 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.875224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.875225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.875272 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.875572 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.875573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.875574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.875690 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:29.875754 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:29.875889 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 15:05:29.875945 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 15:05:29.876060 20563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.870252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:35597 (local address 127.0.0.1:53726)
0806 15:05:29.870299 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:29.870309 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:29.870349 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:29.870462 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:29.870463 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:29.870571 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:29.870602 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:29.870602 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.871417 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.871418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:29.871861 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:29.871862 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.872299 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.872300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:29.872332 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:29.872671 (+ 339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:29.872674 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:29.873684 (+ 1010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:29.875015 (+ 1331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:29.875015 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:29.875016 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:29.875165 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:29.875354 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:29.875355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:29.875355 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:29.875431 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:29.875750 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:29.875751 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:29.875799 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 15:05:29.875871 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 15:05:29.875959 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 15:05:29.877219 20561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bb5ff98bbd640cf9dc4c48628ba98ef: Bootstrap starting.
I20260806 15:05:29.877787 20561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2bb5ff98bbd640cf9dc4c48628ba98ef: Neither blocks nor log segments found. Creating new log.
I20260806 15:05:29.878077 20561 log.cc:824] T 00000000000000000000000000000000 P 2bb5ff98bbd640cf9dc4c48628ba98ef: Log is configured to *not* fsync() on all Append() calls
I20260806 15:05:29.878772 20561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bb5ff98bbd640cf9dc4c48628ba98ef: No bootstrap required, opened a new log
I20260806 15:05:29.880091 20561 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2bb5ff98bbd640cf9dc4c48628ba98ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bb5ff98bbd640cf9dc4c48628ba98ef" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 33083 } } peers { permanent_uuid: "2d2cca91c70c4ea8a7ebc4b90c098967" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 42395 } } peers { permanent_uuid: "d2f132ee0bf94d92901757bc89cde263" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } }
I20260806 15:05:29.880192 20561 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2bb5ff98bbd640cf9dc4c48628ba98ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:05:29.880213 20561 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2bb5ff98bbd640cf9dc4c48628ba98ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bb5ff98bbd640cf9dc4c48628ba98ef, State: Initialized, Role: FOLLOWER
I20260806 15:05:29.880334 20561 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2bb5ff98bbd640cf9dc4c48628ba98ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bb5ff98bbd640cf9dc4c48628ba98ef" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 33083 } } peers { permanent_uuid: "2d2cca91c70c4ea8a7ebc4b90c098967" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 42395 } } peers { permanent_uuid: "d2f132ee0bf94d92901757bc89cde263" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } }
I20260806 15:05:29.880558 20836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bb5ff98bbd640cf9dc4c48628ba98ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bb5ff98bbd640cf9dc4c48628ba98ef" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 33083 } } peers { permanent_uuid: "2d2cca91c70c4ea8a7ebc4b90c098967" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 42395 } } peers { permanent_uuid: "d2f132ee0bf94d92901757bc89cde263" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } } }
I20260806 15:05:29.880614 20561 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2bb5ff98bbd640cf9dc4c48628ba98ef [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:05:29.880635 20836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bb5ff98bbd640cf9dc4c48628ba98ef [sys.catalog]: This master's current role is: FOLLOWER
W20260806 15:05:29.882174 20853 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2bb5ff98bbd640cf9dc4c48628ba98ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:05:29.882385 20853 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:05:29.883019 20460 master_runner.cc:186] Error getting master registration for 127.0.236.125:42395: OK, instance_id { permanent_uuid: "2d2cca91c70c4ea8a7ebc4b90c098967" instance_seqno: 1786028729671190 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 15:05:29.885659 20460 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 15:05:29.902348 20680 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } attempt: 3
Aug 06 15:05:29 dist-test-slave-tc2s krb5kdc[20441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028729, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.125@KRBTEST.COM for kudu/127.0.236.124@KRBTEST.COM
I20260806 15:05:29.909998 20801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.902641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53732 (local address 127.0.236.124:35597)
0806 15:05:29.902686 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:29.902688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:29.902698 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:05:29.902719 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:29.902720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:29.902728 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:29.902785 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:29.903085 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.903796 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.904863 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.905161 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.905223 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:29.907344 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:29.907345 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:29.907346 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:29.907353 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:29.908853 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.909249 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.909250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.909250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.909310 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:29.909497 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:29.909498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:29.909498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:29.909620 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:29.909684 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:29.909815 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 15:05:29.909885 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:05:29.910013 20682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:29.902593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:35597 (local address 127.0.0.1:53732)
0806 15:05:29.902637 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:29.902647 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:29.902681 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:29.902796 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:29.902797 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:29.902967 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:29.903014 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:29.903016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.903870 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.903871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:29.904773 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:29.904776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:29.905255 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:29.905257 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:29.905324 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:29.905780 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:29.905785 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:29.907246 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:29.908932 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:29.908934 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:29.908934 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:29.909150 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:29.909370 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:29.909371 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:29.909371 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:29.909442 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:29.909679 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:29.909679 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:29.909723 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 15:05:29.909788 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 15:05:29.909882 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 15:05:29.911870 20680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d2cca91c70c4ea8a7ebc4b90c098967: Bootstrap starting.
I20260806 15:05:29.912567 20680 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d2cca91c70c4ea8a7ebc4b90c098967: Neither blocks nor log segments found. Creating new log.
I20260806 15:05:29.912874 20680 log.cc:824] T 00000000000000000000000000000000 P 2d2cca91c70c4ea8a7ebc4b90c098967: Log is configured to *not* fsync() on all Append() calls
I20260806 15:05:29.913785 20680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d2cca91c70c4ea8a7ebc4b90c098967: No bootstrap required, opened a new log
I20260806 15:05:29.915728 20680 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d2cca91c70c4ea8a7ebc4b90c098967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bb5ff98bbd640cf9dc4c48628ba98ef" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 33083 } } peers { permanent_uuid: "2d2cca91c70c4ea8a7ebc4b90c098967" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 42395 } } peers { permanent_uuid: "d2f132ee0bf94d92901757bc89cde263" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } }
I20260806 15:05:29.915861 20680 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d2cca91c70c4ea8a7ebc4b90c098967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:05:29.915895 20680 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d2cca91c70c4ea8a7ebc4b90c098967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d2cca91c70c4ea8a7ebc4b90c098967, State: Initialized, Role: FOLLOWER
I20260806 15:05:29.916013 20680 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d2cca91c70c4ea8a7ebc4b90c098967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bb5ff98bbd640cf9dc4c48628ba98ef" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 33083 } } peers { permanent_uuid: "2d2cca91c70c4ea8a7ebc4b90c098967" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 42395 } } peers { permanent_uuid: "d2f132ee0bf94d92901757bc89cde263" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } }
I20260806 15:05:29.916308 20680 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d2cca91c70c4ea8a7ebc4b90c098967 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:05:29.916276 20857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2cca91c70c4ea8a7ebc4b90c098967 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bb5ff98bbd640cf9dc4c48628ba98ef" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 33083 } } peers { permanent_uuid: "2d2cca91c70c4ea8a7ebc4b90c098967" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 42395 } } peers { permanent_uuid: "d2f132ee0bf94d92901757bc89cde263" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } } }
I20260806 15:05:29.916494 20857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2cca91c70c4ea8a7ebc4b90c098967 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 15:05:29.918339 20871 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2d2cca91c70c4ea8a7ebc4b90c098967: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:05:29.918439 20871 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 15:05:29.919099 20608 tablet.cc:2369] T 00000000000000000000000000000000 P 2d2cca91c70c4ea8a7ebc4b90c098967: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:05:29.920200 20572 master_runner.cc:263] No leader master found from master 2d2cca91c70c4ea8a7ebc4b90c098967
I20260806 15:05:29.920253 20572 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 15:05:30.029917 20494 tablet.cc:2369] T 00000000000000000000000000000000 P 2bb5ff98bbd640cf9dc4c48628ba98ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 15:05:30.067399 20722 tablet.cc:2369] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:05:30.142520 20805 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:05:30.142627 20805 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bb5ff98bbd640cf9dc4c48628ba98ef" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 33083 } } peers { permanent_uuid: "2d2cca91c70c4ea8a7ebc4b90c098967" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 42395 } } peers { permanent_uuid: "d2f132ee0bf94d92901757bc89cde263" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } }
I20260806 15:05:30.142863 20805 leader_election.cc:288] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2bb5ff98bbd640cf9dc4c48628ba98ef (127.0.236.126:33083), 2d2cca91c70c4ea8a7ebc4b90c098967 (127.0.236.125:42395)
I20260806 15:05:30.143092 20522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d2f132ee0bf94d92901757bc89cde263" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bb5ff98bbd640cf9dc4c48628ba98ef" is_pre_election: true
I20260806 15:05:30.143133 20636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d2f132ee0bf94d92901757bc89cde263" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d2cca91c70c4ea8a7ebc4b90c098967" is_pre_election: true
I20260806 15:05:30.143229 20522 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2bb5ff98bbd640cf9dc4c48628ba98ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2f132ee0bf94d92901757bc89cde263 in term 0.
I20260806 15:05:30.143252 20636 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2d2cca91c70c4ea8a7ebc4b90c098967 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2f132ee0bf94d92901757bc89cde263 in term 0.
I20260806 15:05:30.143445 20719 leader_election.cc:302] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2cca91c70c4ea8a7ebc4b90c098967, d2f132ee0bf94d92901757bc89cde263; no voters:
I20260806 15:05:30.143582 20805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:05:30.143648 20805 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:05:30.143664 20805 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:05:30.144364 20805 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bb5ff98bbd640cf9dc4c48628ba98ef" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 33083 } } peers { permanent_uuid: "2d2cca91c70c4ea8a7ebc4b90c098967" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 42395 } } peers { permanent_uuid: "d2f132ee0bf94d92901757bc89cde263" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } }
I20260806 15:05:30.144481 20805 leader_election.cc:288] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [CANDIDATE]: Term 1 election: Requested vote from peers 2bb5ff98bbd640cf9dc4c48628ba98ef (127.0.236.126:33083), 2d2cca91c70c4ea8a7ebc4b90c098967 (127.0.236.125:42395)
I20260806 15:05:30.144614 20636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d2f132ee0bf94d92901757bc89cde263" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d2cca91c70c4ea8a7ebc4b90c098967"
I20260806 15:05:30.144637 20522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d2f132ee0bf94d92901757bc89cde263" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bb5ff98bbd640cf9dc4c48628ba98ef"
I20260806 15:05:30.144688 20636 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d2cca91c70c4ea8a7ebc4b90c098967 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:05:30.144698 20522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2bb5ff98bbd640cf9dc4c48628ba98ef [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:05:30.145488 20636 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2d2cca91c70c4ea8a7ebc4b90c098967 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2f132ee0bf94d92901757bc89cde263 in term 1.
I20260806 15:05:30.145648 20719 leader_election.cc:302] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2cca91c70c4ea8a7ebc4b90c098967, d2f132ee0bf94d92901757bc89cde263; no voters:
I20260806 15:05:30.145587 20522 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2bb5ff98bbd640cf9dc4c48628ba98ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2f132ee0bf94d92901757bc89cde263 in term 1.
I20260806 15:05:30.145740 20805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:05:30.145910 20805 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [term 1 LEADER]: Becoming Leader. State: Replica: d2f132ee0bf94d92901757bc89cde263, State: Running, Role: LEADER
I20260806 15:05:30.146010 20805 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bb5ff98bbd640cf9dc4c48628ba98ef" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 33083 } } peers { permanent_uuid: "2d2cca91c70c4ea8a7ebc4b90c098967" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 42395 } } peers { permanent_uuid: "d2f132ee0bf94d92901757bc89cde263" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } }
I20260806 15:05:30.146401 20876 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2f132ee0bf94d92901757bc89cde263. Latest consensus state: current_term: 1 leader_uuid: "d2f132ee0bf94d92901757bc89cde263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bb5ff98bbd640cf9dc4c48628ba98ef" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 33083 } } peers { permanent_uuid: "2d2cca91c70c4ea8a7ebc4b90c098967" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 42395 } } peers { permanent_uuid: "d2f132ee0bf94d92901757bc89cde263" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } } }
I20260806 15:05:30.146489 20876 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [sys.catalog]: This master's current role is: LEADER
I20260806 15:05:30.146672 20879 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:05:30.146835 20879 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:05:30.148108 20636 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2d2cca91c70c4ea8a7ebc4b90c098967 [term 1 FOLLOWER]: Refusing update from remote peer d2f132ee0bf94d92901757bc89cde263: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:05:30.148110 20522 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2bb5ff98bbd640cf9dc4c48628ba98ef [term 1 FOLLOWER]: Refusing update from remote peer d2f132ee0bf94d92901757bc89cde263: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:05:30.148381 20805 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2cca91c70c4ea8a7ebc4b90c098967" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 42395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:05:30.148490 20876 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb5ff98bbd640cf9dc4c48628ba98ef" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 33083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:05:30.149446 20836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bb5ff98bbd640cf9dc4c48628ba98ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2f132ee0bf94d92901757bc89cde263. Latest consensus state: current_term: 1 leader_uuid: "d2f132ee0bf94d92901757bc89cde263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bb5ff98bbd640cf9dc4c48628ba98ef" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 33083 } } peers { permanent_uuid: "2d2cca91c70c4ea8a7ebc4b90c098967" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 42395 } } peers { permanent_uuid: "d2f132ee0bf94d92901757bc89cde263" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } } }
I20260806 15:05:30.149528 20836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bb5ff98bbd640cf9dc4c48628ba98ef [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:05:30.149519 20857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2cca91c70c4ea8a7ebc4b90c098967 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2f132ee0bf94d92901757bc89cde263. Latest consensus state: current_term: 1 leader_uuid: "d2f132ee0bf94d92901757bc89cde263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bb5ff98bbd640cf9dc4c48628ba98ef" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 33083 } } peers { permanent_uuid: "2d2cca91c70c4ea8a7ebc4b90c098967" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 42395 } } peers { permanent_uuid: "d2f132ee0bf94d92901757bc89cde263" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } } }
I20260806 15:05:30.149638 20857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2cca91c70c4ea8a7ebc4b90c098967 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:05:30.150344 20805 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d2f132ee0bf94d92901757bc89cde263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bb5ff98bbd640cf9dc4c48628ba98ef" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 33083 } } peers { permanent_uuid: "2d2cca91c70c4ea8a7ebc4b90c098967" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 42395 } } peers { permanent_uuid: "d2f132ee0bf94d92901757bc89cde263" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } } }
I20260806 15:05:30.150424 20805 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [sys.catalog]: This master's current role is: LEADER
I20260806 15:05:30.150455 20805 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d2f132ee0bf94d92901757bc89cde263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bb5ff98bbd640cf9dc4c48628ba98ef" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 33083 } } peers { permanent_uuid: "2d2cca91c70c4ea8a7ebc4b90c098967" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 42395 } } peers { permanent_uuid: "d2f132ee0bf94d92901757bc89cde263" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } } }
I20260806 15:05:30.150496 20805 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263 [sys.catalog]: This master's current role is: LEADER
I20260806 15:05:30.150547 20881 mvcc.cc:205] Tried to move back new op lower bound from 7315573678684495872 to 7315573678678605824. Current Snapshot: MvccSnapshot[applied={T|T < 7315573678684495872}]
I20260806 15:05:30.150609 20836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bb5ff98bbd640cf9dc4c48628ba98ef [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d2f132ee0bf94d92901757bc89cde263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bb5ff98bbd640cf9dc4c48628ba98ef" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 33083 } } peers { permanent_uuid: "2d2cca91c70c4ea8a7ebc4b90c098967" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 42395 } } peers { permanent_uuid: "d2f132ee0bf94d92901757bc89cde263" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } } }
I20260806 15:05:30.150691 20882 mvcc.cc:205] Tried to move back new op lower bound from 7315573678684495872 to 7315573678678605824. Current Snapshot: MvccSnapshot[applied={T|T < 7315573678684495872}]
I20260806 15:05:30.150704 20836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bb5ff98bbd640cf9dc4c48628ba98ef [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:05:30.150907 20857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2cca91c70c4ea8a7ebc4b90c098967 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d2f132ee0bf94d92901757bc89cde263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bb5ff98bbd640cf9dc4c48628ba98ef" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 33083 } } peers { permanent_uuid: "2d2cca91c70c4ea8a7ebc4b90c098967" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 42395 } } peers { permanent_uuid: "d2f132ee0bf94d92901757bc89cde263" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 35597 } } }
I20260806 15:05:30.150992 20857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2cca91c70c4ea8a7ebc4b90c098967 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:05:30.151158 20883 mvcc.cc:205] Tried to move back new op lower bound from 7315573678684495872 to 7315573678678605824. Current Snapshot: MvccSnapshot[applied={T|T < 7315573678684495872 or (T in {7315573678684495872})}]
I20260806 15:05:30.151371 20879 catalog_manager.cc:1347] Generated new cluster ID: e8b233f0016840eaa637c7ee5227a319
I20260806 15:05:30.151412 20879 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:05:30.157006 20879 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:05:30.157509 20879 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:05:30.164561 20879 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2f132ee0bf94d92901757bc89cde263: Generated new TSK 0
I20260806 15:05:30.164723 20879 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:05:30.882817 20853 catalog_manager.cc:1259] Loaded cluster ID: e8b233f0016840eaa637c7ee5227a319
I20260806 15:05:30.882891 20853 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2bb5ff98bbd640cf9dc4c48628ba98ef: loading cluster ID for follower catalog manager: success
I20260806 15:05:30.883728 20853 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2bb5ff98bbd640cf9dc4c48628ba98ef: acquiring CA information for follower catalog manager: success
I20260806 15:05:30.884030 20853 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2bb5ff98bbd640cf9dc4c48628ba98ef: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 15:05:30.918870 20871 catalog_manager.cc:1259] Loaded cluster ID: e8b233f0016840eaa637c7ee5227a319
I20260806 15:05:30.918929 20871 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2d2cca91c70c4ea8a7ebc4b90c098967: loading cluster ID for follower catalog manager: success
I20260806 15:05:30.919728 20871 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2d2cca91c70c4ea8a7ebc4b90c098967: acquiring CA information for follower catalog manager: success
I20260806 15:05:30.920017 20871 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2d2cca91c70c4ea8a7ebc4b90c098967: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 15:05:31.212195 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 20460
I20260806 15:05:31.227401 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 20572
I20260806 15:05:31.235283 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 20689
2026-08-06T15:05:31Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3060 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:05:31 dist-test-slave-tc2s krb5kdc[20902](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:05:31 dist-test-slave-tc2s krb5kdc[20902](info): set up 2 sockets
Aug 06 15:05:31 dist-test-slave-tc2s krb5kdc[20902](info): commencing operation
krb5kdc: starting...
W20260806 15:05:32.349998 945 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:05:32 dist-test-slave-tc2s krb5kdc[20902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:05:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:05:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:05:32.457672 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:43319
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:34903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:43319
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:05:32.536901 20923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:32.537101 20923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:32.537122 20923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:32.538210 20923 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:05:32.538245 20923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:32.538259 20923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:32.538269 20923 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:05:32.538285 20923 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:05:32.539666 20923 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:34903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:43319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:43319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:32.539842 20923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:32.540012 20923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:05:32.544416 20934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:32.544442 20930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:32.544417 20929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:32.544718 20923 server_base.cc:1034] running on GCE node
I20260806 15:05:32.544970 20923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:32.545235 20923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:32.546375 20923 hybrid_clock.cc:648] HybridClock initialized: now 1786028732546361 us; error 25 us; skew 500 ppm
Aug 06 15:05:32 dist-test-slave-tc2s krb5kdc[20902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028732, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:32.548609 20923 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:05:32.548995 20923 webserver.cc:460] Webserver started at http://127.0.236.126:38391/ using document root <none> and password file <none>
I20260806 15:05:32.549180 20923 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:32.549227 20923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:05:32.549357 20923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:05:32.550411 20923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "587f1fc196ed4c45abd21145b9db11c8"
format_stamp: "Formatted at 2026-08-06 15:05:32 on dist-test-slave-tc2s"
server_key: "a1d24a50ac7735dcd0bdb702813d76f4"
server_key_iv: "a7e61c70b2a739223b55938102d4c1ba"
server_key_version: "encryptionkey@0"
I20260806 15:05:32.550729 20923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "587f1fc196ed4c45abd21145b9db11c8"
format_stamp: "Formatted at 2026-08-06 15:05:32 on dist-test-slave-tc2s"
server_key: "a1d24a50ac7735dcd0bdb702813d76f4"
server_key_iv: "a7e61c70b2a739223b55938102d4c1ba"
server_key_version: "encryptionkey@0"
I20260806 15:05:32.551961 20923 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:05:32.552601 20945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:32.552747 20923 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:05:32.552830 20923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "587f1fc196ed4c45abd21145b9db11c8"
format_stamp: "Formatted at 2026-08-06 15:05:32 on dist-test-slave-tc2s"
server_key: "a1d24a50ac7735dcd0bdb702813d76f4"
server_key_iv: "a7e61c70b2a739223b55938102d4c1ba"
server_key_version: "encryptionkey@0"
I20260806 15:05:32.552889 20923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:32.592144 20923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:32.593806 20923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:32.593943 20923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:32.597647 20923 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:43319
I20260806 15:05:32.597694 21027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:43319 every 8 connection(s)
I20260806 15:05:32.598006 20923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:05:32.598893 21028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:05:32.601059 21028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 587f1fc196ed4c45abd21145b9db11c8: Bootstrap starting.
I20260806 15:05:32.601651 21028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 587f1fc196ed4c45abd21145b9db11c8: Neither blocks nor log segments found. Creating new log.
I20260806 15:05:32.601771 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 20923
I20260806 15:05:32.601886 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:05:32.602026 21028 log.cc:824] T 00000000000000000000000000000000 P 587f1fc196ed4c45abd21145b9db11c8: Log is configured to *not* fsync() on all Append() calls
I20260806 15:05:32.602123 945 external_mini_cluster.cc:1428] Setting key 8bf8607a865d1ff6fa979d28ab175cde
I20260806 15:05:32.602798 21028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 587f1fc196ed4c45abd21145b9db11c8: No bootstrap required, opened a new log
I20260806 15:05:32.605057 21028 raft_consensus.cc:348] T 00000000000000000000000000000000 P 587f1fc196ed4c45abd21145b9db11c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587f1fc196ed4c45abd21145b9db11c8" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43319 } }
I20260806 15:05:32.605223 21028 raft_consensus.cc:374] T 00000000000000000000000000000000 P 587f1fc196ed4c45abd21145b9db11c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:05:32.605268 21028 raft_consensus.cc:729] T 00000000000000000000000000000000 P 587f1fc196ed4c45abd21145b9db11c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 587f1fc196ed4c45abd21145b9db11c8, State: Initialized, Role: FOLLOWER
I20260806 15:05:32.605397 21028 consensus_queue.cc:260] T 00000000000000000000000000000000 P 587f1fc196ed4c45abd21145b9db11c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587f1fc196ed4c45abd21145b9db11c8" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43319 } }
I20260806 15:05:32.605473 21028 raft_consensus.cc:388] T 00000000000000000000000000000000 P 587f1fc196ed4c45abd21145b9db11c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:05:32.605507 21028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 587f1fc196ed4c45abd21145b9db11c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:05:32.605548 21028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 587f1fc196ed4c45abd21145b9db11c8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:05:32.606173 21028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 587f1fc196ed4c45abd21145b9db11c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587f1fc196ed4c45abd21145b9db11c8" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43319 } }
I20260806 15:05:32.606273 21028 leader_election.cc:302] T 00000000000000000000000000000000 P 587f1fc196ed4c45abd21145b9db11c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 587f1fc196ed4c45abd21145b9db11c8; no voters:
I20260806 15:05:32.606451 21028 leader_election.cc:288] T 00000000000000000000000000000000 P 587f1fc196ed4c45abd21145b9db11c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:05:32.606504 21034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 587f1fc196ed4c45abd21145b9db11c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:05:32.606755 21028 sys_catalog.cc:564] T 00000000000000000000000000000000 P 587f1fc196ed4c45abd21145b9db11c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:05:32.606915 21034 raft_consensus.cc:686] T 00000000000000000000000000000000 P 587f1fc196ed4c45abd21145b9db11c8 [term 1 LEADER]: Becoming Leader. State: Replica: 587f1fc196ed4c45abd21145b9db11c8, State: Running, Role: LEADER
I20260806 15:05:32.607026 21034 consensus_queue.cc:237] T 00000000000000000000000000000000 P 587f1fc196ed4c45abd21145b9db11c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587f1fc196ed4c45abd21145b9db11c8" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43319 } }
I20260806 15:05:32.607343 21034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 587f1fc196ed4c45abd21145b9db11c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 587f1fc196ed4c45abd21145b9db11c8. Latest consensus state: current_term: 1 leader_uuid: "587f1fc196ed4c45abd21145b9db11c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587f1fc196ed4c45abd21145b9db11c8" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43319 } } }
I20260806 15:05:32.607427 21034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 587f1fc196ed4c45abd21145b9db11c8 [sys.catalog]: This master's current role is: LEADER
I20260806 15:05:32.607352 21035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 587f1fc196ed4c45abd21145b9db11c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "587f1fc196ed4c45abd21145b9db11c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587f1fc196ed4c45abd21145b9db11c8" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43319 } } }
I20260806 15:05:32.607517 21035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 587f1fc196ed4c45abd21145b9db11c8 [sys.catalog]: This master's current role is: LEADER
Aug 06 15:05:32 dist-test-slave-tc2s krb5kdc[20902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
W20260806 15:05:32.609079 21061 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 587f1fc196ed4c45abd21145b9db11c8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:05:32.609175 21061 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:05:32.609289 21046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:05:32.609470 21046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:05:32.610880 21046 catalog_manager.cc:1347] Generated new cluster ID: bfd625df61b143b7a97d71ac3f65da40
I20260806 15:05:32.610931 21046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:05:32.612490 21032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:32.602557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60468 (local address 127.0.236.126:43319)
0806 15:05:32.602703 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:32.602713 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:32.602736 (+ 23us) server_negotiation.cc:407] Connection header received
0806 15:05:32.603159 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:32.603165 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:32.603209 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:32.603314 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:32.604051 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:32.604901 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:32.605920 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:32.606344 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:32.606453 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:32.609024 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:32.609028 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:32.609028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:32.609039 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:32.610938 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:32.611340 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:32.611343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:32.611344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:32.611446 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:32.611647 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:32.611648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:32.611648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:32.611911 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:32.612040 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:32.612172 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 15:05:32.612293 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
I20260806 15:05:32.613265 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:05:32.617216 21046 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:05:32.617877 21046 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:05:32.627564 21046 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 587f1fc196ed4c45abd21145b9db11c8: Generated new TSK 0
I20260806 15:05:32.627717 21046 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:05:32.666657 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 20923
2026-08-06T15:05:32Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1433 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:05:32 dist-test-slave-tc2s krb5kdc[21069](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:05:32 dist-test-slave-tc2s krb5kdc[21069](info): set up 2 sockets
Aug 06 15:05:32 dist-test-slave-tc2s krb5kdc[21069](info): commencing operation
krb5kdc: starting...
W20260806 15:05:33.780354 945 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:05:33 dist-test-slave-tc2s krb5kdc[21069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:05:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:05:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:05:33.889526 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:39761
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:35035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:39761
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:05:33.968024 21087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:33.968204 21087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:33.968223 21087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:33.969319 21087 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:05:33.969364 21087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:33.969378 21087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:33.969403 21087 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:05:33.969425 21087 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:05:33.970923 21087 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:35035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:39761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:39761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:33.971131 21087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:33.971326 21087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:05:33.975693 21100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:33.975723 21099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:33.975731 21107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:33.975771 21087 server_base.cc:1034] running on GCE node
I20260806 15:05:33.976089 21087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:33.976353 21087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:33.977490 21087 hybrid_clock.cc:648] HybridClock initialized: now 1786028733977481 us; error 27 us; skew 500 ppm
Aug 06 15:05:33 dist-test-slave-tc2s krb5kdc[21069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028733, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:33.979722 21087 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:05:33.980095 21087 webserver.cc:460] Webserver started at http://127.0.236.126:40577/ using document root <none> and password file <none>
I20260806 15:05:33.980301 21087 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:33.980360 21087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:05:33.980492 21087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:05:33.981421 21087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "e58f8c49b33846ea9e314f0723da7a9f"
format_stamp: "Formatted at 2026-08-06 15:05:33 on dist-test-slave-tc2s"
server_key: "5d9a00fe1a4ab734dae8286e929fbb7f"
server_key_iv: "43e6c129b190aab570e4fb18c8076d55"
server_key_version: "encryptionkey@0"
I20260806 15:05:33.981743 21087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "e58f8c49b33846ea9e314f0723da7a9f"
format_stamp: "Formatted at 2026-08-06 15:05:33 on dist-test-slave-tc2s"
server_key: "5d9a00fe1a4ab734dae8286e929fbb7f"
server_key_iv: "43e6c129b190aab570e4fb18c8076d55"
server_key_version: "encryptionkey@0"
I20260806 15:05:33.982964 21087 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:05:33.983633 21116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:33.983774 21087 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:05:33.983856 21087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "e58f8c49b33846ea9e314f0723da7a9f"
format_stamp: "Formatted at 2026-08-06 15:05:33 on dist-test-slave-tc2s"
server_key: "5d9a00fe1a4ab734dae8286e929fbb7f"
server_key_iv: "43e6c129b190aab570e4fb18c8076d55"
server_key_version: "encryptionkey@0"
I20260806 15:05:33.983922 21087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:33.998127 21087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:33.999794 21087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:33.999917 21087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:34.003624 21087 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:39761
I20260806 15:05:34.003659 21194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:39761 every 8 connection(s)
I20260806 15:05:34.003953 21087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:05:34.004904 21195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:05:34.007042 21195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f: Bootstrap starting.
I20260806 15:05:34.007587 21195 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f: Neither blocks nor log segments found. Creating new log.
I20260806 15:05:34.007844 21195 log.cc:824] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f: Log is configured to *not* fsync() on all Append() calls
I20260806 15:05:34.008466 21195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f: No bootstrap required, opened a new log
I20260806 15:05:34.010008 21195 raft_consensus.cc:348] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e58f8c49b33846ea9e314f0723da7a9f" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39761 } }
I20260806 15:05:34.010135 21195 raft_consensus.cc:374] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:05:34.010167 21195 raft_consensus.cc:729] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e58f8c49b33846ea9e314f0723da7a9f, State: Initialized, Role: FOLLOWER
I20260806 15:05:34.010296 21195 consensus_queue.cc:260] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e58f8c49b33846ea9e314f0723da7a9f" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39761 } }
I20260806 15:05:34.010354 21195 raft_consensus.cc:388] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:05:34.010376 21195 raft_consensus.cc:482] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:05:34.010429 21195 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:05:34.011161 21195 raft_consensus.cc:504] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e58f8c49b33846ea9e314f0723da7a9f" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39761 } }
I20260806 15:05:34.011252 21195 leader_election.cc:302] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e58f8c49b33846ea9e314f0723da7a9f; no voters:
I20260806 15:05:34.011404 21195 leader_election.cc:288] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:05:34.011431 21201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:05:34.011566 21201 raft_consensus.cc:686] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [term 1 LEADER]: Becoming Leader. State: Replica: e58f8c49b33846ea9e314f0723da7a9f, State: Running, Role: LEADER
I20260806 15:05:34.011633 21195 sys_catalog.cc:564] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:05:34.011650 21201 consensus_queue.cc:237] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e58f8c49b33846ea9e314f0723da7a9f" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39761 } }
I20260806 15:05:34.012032 21203 sys_catalog.cc:455] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e58f8c49b33846ea9e314f0723da7a9f. Latest consensus state: current_term: 1 leader_uuid: "e58f8c49b33846ea9e314f0723da7a9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e58f8c49b33846ea9e314f0723da7a9f" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39761 } } }
I20260806 15:05:34.012117 21203 sys_catalog.cc:458] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [sys.catalog]: This master's current role is: LEADER
I20260806 15:05:34.012022 21202 sys_catalog.cc:455] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e58f8c49b33846ea9e314f0723da7a9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e58f8c49b33846ea9e314f0723da7a9f" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39761 } } }
I20260806 15:05:34.012250 21202 sys_catalog.cc:458] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [sys.catalog]: This master's current role is: LEADER
I20260806 15:05:34.012334 21209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:05:34.012859 21209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:05:34.013615 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 21087
I20260806 15:05:34.013738 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:05:34.013960 945 external_mini_cluster.cc:1428] Setting key 77b02ad430609d1ef0c20244b8b59155
I20260806 15:05:34.014829 21209 catalog_manager.cc:1347] Generated new cluster ID: 43da28a2a2de450ead475558254a73e2
I20260806 15:05:34.014873 21209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:05:34.018195 21209 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:05:34.018819 21209 catalog_manager.cc:1504] Loading token signing keys...
Aug 06 15:05:34 dist-test-slave-tc2s krb5kdc[21069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:05:34.022838 21209 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f: Generated new TSK 0
I20260806 15:05:34.023032 21209 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:05:34.025213 21230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:34.014405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44554 (local address 127.0.236.126:39761)
0806 15:05:34.015101 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:34.015110 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:34.015142 (+ 32us) server_negotiation.cc:407] Connection header received
0806 15:05:34.015249 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:34.015255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:34.015308 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:34.015442 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:34.016140 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.017590 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.018728 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.019363 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.019492 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:34.022158 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:34.022162 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:34.022162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:34.022174 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:34.023873 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:34.024180 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:34.024183 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:34.024184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:34.024268 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:34.024442 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:34.024443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:34.024443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:34.024650 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:34.024785 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:34.024892 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 15:05:34.025044 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":645,"thread_start_us":54,"threads_started":1}
WARNING: no policy specified for kudu/127.0.236.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
I20260806 15:05:34.049861 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:0
--local_ip_for_outbound_sockets=127.0.236.65
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:39761
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:35035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:05:34.129594 21236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:34.129856 21236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:34.129923 21236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:34.131052 21236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:34.131093 21236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:34.131127 21236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:05:34.132541 21236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:35035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:39761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:34.132752 21236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:34.132952 21236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:05:34.137565 21246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:34.137583 21236 server_base.cc:1034] running on GCE node
W20260806 15:05:34.137540 21244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:34.137552 21242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:34.138020 21236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:34.138243 21236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:34.139379 21236 hybrid_clock.cc:648] HybridClock initialized: now 1786028734139364 us; error 28 us; skew 500 ppm
Aug 06 15:05:34 dist-test-slave-tc2s krb5kdc[21069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028734, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:34.141611 21236 init.cc:377] Logged in from keytab as kudu/127.0.236.65@KRBTEST.COM (short username kudu)
I20260806 15:05:34.142038 21236 webserver.cc:460] Webserver started at http://127.0.236.65:37915/ using document root <none> and password file <none>
I20260806 15:05:34.142221 21236 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:34.142267 21236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:05:34.142392 21236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:05:34.143321 21236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/data/instance:
uuid: "5e08410ceb93456ea29bb3eb223bd15f"
format_stamp: "Formatted at 2026-08-06 15:05:34 on dist-test-slave-tc2s"
server_key: "77e5c0221ea6f3457fe584f6278b8acf"
server_key_iv: "958e45693423899335221e41aeaa23db"
server_key_version: "encryptionkey@0"
I20260806 15:05:34.143632 21236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/wal/instance:
uuid: "5e08410ceb93456ea29bb3eb223bd15f"
format_stamp: "Formatted at 2026-08-06 15:05:34 on dist-test-slave-tc2s"
server_key: "77e5c0221ea6f3457fe584f6278b8acf"
server_key_iv: "958e45693423899335221e41aeaa23db"
server_key_version: "encryptionkey@0"
I20260806 15:05:34.144778 21236 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:05:34.145375 21255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:34.145522 21236 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:05:34.145599 21236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "5e08410ceb93456ea29bb3eb223bd15f"
format_stamp: "Formatted at 2026-08-06 15:05:34 on dist-test-slave-tc2s"
server_key: "77e5c0221ea6f3457fe584f6278b8acf"
server_key_iv: "958e45693423899335221e41aeaa23db"
server_key_version: "encryptionkey@0"
I20260806 15:05:34.145663 21236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:34.152297 21236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:34.153834 21236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:34.154012 21236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:34.154201 21236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:05:34.154465 21236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:05:34.154497 21236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:34.154523 21236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:05:34.154541 21236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:34.160106 21236 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:34465
I20260806 15:05:34.160192 21429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:34465 every 8 connection(s)
I20260806 15:05:34.160498 21236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:05:34.164014 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 21236
I20260806 15:05:34.164127 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/wal/instance
I20260806 15:05:34.164294 945 external_mini_cluster.cc:1428] Setting key 5dcfea08348cd96f55cfaedc0da1a0e5
Aug 06 15:05:34 dist-test-slave-tc2s krb5kdc[21069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028734, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:05:34.168928 21230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:34.161119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:54749 (local address 127.0.236.126:39761)
0806 15:05:34.161169 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:34.161172 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:34.161345 (+ 173us) server_negotiation.cc:407] Connection header received
0806 15:05:34.161749 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:34.161750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:34.161760 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:34.161811 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:34.162682 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.163174 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.164105 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.164369 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.164429 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:34.166355 (+ 1926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:34.166356 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:34.166357 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:34.166365 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:34.167610 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:34.168031 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:34.168031 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:34.168031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:34.168079 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:34.168309 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:34.168310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:34.168310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:34.168397 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:34.168443 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:34.168755 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 15:05:34.168835 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 15:05:34.169121 21432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:34.161097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.65:54749)
0806 15:05:34.161240 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:34.161271 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:34.161670 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:34.161861 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:34.161864 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:34.162434 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:34.162610 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:34.162614 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.163237 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.163237 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:34.164018 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:34.164020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.164436 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.164437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:34.164563 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:34.164994 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:34.165002 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:34.166268 (+ 1266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:34.167689 (+ 1421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:34.167696 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:34.167698 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:34.167968 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:34.168129 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:34.168130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:34.168131 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:34.168258 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:34.168449 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:34.168454 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:34.168629 (+ 175us) client_negotiation.cc:769] Sending connection context
0806 15:05:34.168746 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 15:05:34.168907 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":83,"threads_started":1}
I20260806 15:05:34.169615 21430 heartbeater.cc:347] Connected to a master server at 127.0.236.126:39761
I20260806 15:05:34.169704 21430 heartbeater.cc:464] Registering TS with master...
I20260806 15:05:34.169858 21430 heartbeater.cc:511] Master 127.0.236.126:39761 requested a full tablet report, sending...
I20260806 15:05:34.170356 21139 ts_manager.cc:194] Registered new tserver with Master: 5e08410ceb93456ea29bb3eb223bd15f (127.0.236.65:34465)
I20260806 15:05:34.170923 21139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.65@KRBTEST.COM'} at 127.0.236.65:54749
WARNING: no policy specified for kudu/127.0.236.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
I20260806 15:05:34.186580 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:0
--local_ip_for_outbound_sockets=127.0.236.66
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:39761
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:35035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:05:34.264834 21437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:34.265026 21437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:34.265058 21437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:34.266193 21437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:34.266230 21437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:34.266265 21437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:05:34.267663 21437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:35035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:39761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:34.267879 21437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:34.268075 21437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:05:34.272680 21447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:34.272672 21454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:34.272680 21448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:34.272861 21437 server_base.cc:1034] running on GCE node
I20260806 15:05:34.273032 21437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:34.273231 21437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:34.274361 21437 hybrid_clock.cc:648] HybridClock initialized: now 1786028734274350 us; error 27 us; skew 500 ppm
Aug 06 15:05:34 dist-test-slave-tc2s krb5kdc[21069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028734, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:34.276588 21437 init.cc:377] Logged in from keytab as kudu/127.0.236.66@KRBTEST.COM (short username kudu)
I20260806 15:05:34.276961 21437 webserver.cc:460] Webserver started at http://127.0.236.66:33159/ using document root <none> and password file <none>
I20260806 15:05:34.277161 21437 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:34.277204 21437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:05:34.277323 21437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:05:34.278299 21437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/data/instance:
uuid: "0be9946ecc994c248f9ac8a34da06763"
format_stamp: "Formatted at 2026-08-06 15:05:34 on dist-test-slave-tc2s"
server_key: "bc6eee00b36ea72367bd00dfdd03aeff"
server_key_iv: "7a5c48eb9584ca18f4053cd9878ab378"
server_key_version: "encryptionkey@0"
I20260806 15:05:34.278604 21437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/wal/instance:
uuid: "0be9946ecc994c248f9ac8a34da06763"
format_stamp: "Formatted at 2026-08-06 15:05:34 on dist-test-slave-tc2s"
server_key: "bc6eee00b36ea72367bd00dfdd03aeff"
server_key_iv: "7a5c48eb9584ca18f4053cd9878ab378"
server_key_version: "encryptionkey@0"
I20260806 15:05:34.279749 21437 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:05:34.280396 21464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:34.280546 21437 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:05:34.280622 21437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "0be9946ecc994c248f9ac8a34da06763"
format_stamp: "Formatted at 2026-08-06 15:05:34 on dist-test-slave-tc2s"
server_key: "bc6eee00b36ea72367bd00dfdd03aeff"
server_key_iv: "7a5c48eb9584ca18f4053cd9878ab378"
server_key_version: "encryptionkey@0"
I20260806 15:05:34.280684 21437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:34.292728 21437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:34.294694 21437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:34.294818 21437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:34.295008 21437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:05:34.295271 21437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:05:34.295303 21437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:34.295329 21437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:05:34.295353 21437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:34.300587 21437 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:34101
I20260806 15:05:34.300647 21632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:34101 every 8 connection(s)
I20260806 15:05:34.300892 21437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
Aug 06 15:05:34 dist-test-slave-tc2s krb5kdc[21069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028734, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:05:34.308684 21230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:34.301503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:56857 (local address 127.0.236.126:39761)
0806 15:05:34.301581 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:34.301584 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:34.301756 (+ 172us) server_negotiation.cc:407] Connection header received
0806 15:05:34.302236 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:34.302237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:34.302248 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:34.302295 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:34.303095 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.303576 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.304504 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.304678 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.304718 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:34.306576 (+ 1858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:34.306578 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:34.306578 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:34.306584 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:34.307618 (+ 1034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:34.307953 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:34.307954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:34.307954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:34.307999 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:34.308163 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:34.308164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:34.308164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:34.308253 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:34.308296 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:34.308523 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 15:05:34.308600 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 15:05:34.308815 21638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:34.301478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.66:56857)
0806 15:05:34.301646 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:34.301680 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:34.302156 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:34.302343 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:34.302346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:34.302855 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:34.303032 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:34.303036 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.303661 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.303662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:34.304431 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:34.304433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.304727 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.304728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:34.304818 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:34.305165 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:34.305172 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:34.306495 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:34.307681 (+ 1186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:34.307686 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:34.307687 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:34.307886 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:34.308040 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:34.308041 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:34.308041 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:34.308124 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:34.308292 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:34.308295 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:34.308392 (+ 97us) client_negotiation.cc:769] Sending connection context
0806 15:05:34.308490 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 15:05:34.308625 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
I20260806 15:05:34.309304 21633 heartbeater.cc:347] Connected to a master server at 127.0.236.126:39761
I20260806 15:05:34.309388 21633 heartbeater.cc:464] Registering TS with master...
I20260806 15:05:34.309553 21633 heartbeater.cc:511] Master 127.0.236.126:39761 requested a full tablet report, sending...
I20260806 15:05:34.309935 21139 ts_manager.cc:194] Registered new tserver with Master: 0be9946ecc994c248f9ac8a34da06763 (127.0.236.66:34101)
I20260806 15:05:34.310279 21139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.66@KRBTEST.COM'} at 127.0.236.66:56857
I20260806 15:05:34.310698 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 21437
I20260806 15:05:34.310809 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/wal/instance
I20260806 15:05:34.310998 945 external_mini_cluster.cc:1428] Setting key 9644c42a99448d094d972af5f72984d5
WARNING: no policy specified for kudu/127.0.236.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
I20260806 15:05:34.333861 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:0
--local_ip_for_outbound_sockets=127.0.236.67
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:39761
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:35035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:05:34.412178 21641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:34.412405 21641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:34.412439 21641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:34.413581 21641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:34.413621 21641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:34.413650 21641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:05:34.415115 21641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:35035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:39761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:34.415323 21641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:34.415534 21641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:05:34.420236 21652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:34.420209 21649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:34.420243 21641 server_base.cc:1034] running on GCE node
W20260806 15:05:34.420214 21650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:34.420619 21641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:34.420849 21641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:34.421942 21641 hybrid_clock.cc:648] HybridClock initialized: now 1786028734421930 us; error 30 us; skew 500 ppm
Aug 06 15:05:34 dist-test-slave-tc2s krb5kdc[21069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028734, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:34.424237 21641 init.cc:377] Logged in from keytab as kudu/127.0.236.67@KRBTEST.COM (short username kudu)
I20260806 15:05:34.424651 21641 webserver.cc:460] Webserver started at http://127.0.236.67:37337/ using document root <none> and password file <none>
I20260806 15:05:34.424868 21641 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:34.424914 21641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:05:34.425040 21641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:05:34.426055 21641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/data/instance:
uuid: "925546e3fb0c4e378b64d8715107c0ef"
format_stamp: "Formatted at 2026-08-06 15:05:34 on dist-test-slave-tc2s"
server_key: "98516631d3b8cc386d30c57e1567eee6"
server_key_iv: "e2b373d9074aecd6cbccff6bf3f5c53e"
server_key_version: "encryptionkey@0"
I20260806 15:05:34.426373 21641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/wal/instance:
uuid: "925546e3fb0c4e378b64d8715107c0ef"
format_stamp: "Formatted at 2026-08-06 15:05:34 on dist-test-slave-tc2s"
server_key: "98516631d3b8cc386d30c57e1567eee6"
server_key_iv: "e2b373d9074aecd6cbccff6bf3f5c53e"
server_key_version: "encryptionkey@0"
I20260806 15:05:34.427551 21641 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:05:34.428189 21659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:34.428331 21641 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:05:34.428421 21641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "925546e3fb0c4e378b64d8715107c0ef"
format_stamp: "Formatted at 2026-08-06 15:05:34 on dist-test-slave-tc2s"
server_key: "98516631d3b8cc386d30c57e1567eee6"
server_key_iv: "e2b373d9074aecd6cbccff6bf3f5c53e"
server_key_version: "encryptionkey@0"
I20260806 15:05:34.428481 21641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:34.436730 21641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:34.438328 21641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:34.438458 21641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:34.438647 21641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:05:34.438908 21641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:05:34.438942 21641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:34.438967 21641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:05:34.438985 21641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:34.444304 21641 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:46099
I20260806 15:05:34.444339 21850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:46099 every 8 connection(s)
I20260806 15:05:34.444598 21641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
I20260806 15:05:34.447672 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 21641
I20260806 15:05:34.447791 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/wal/instance
I20260806 15:05:34.447948 945 external_mini_cluster.cc:1428] Setting key b27b4c1bf992e612471aef543f4dc4cc
Aug 06 15:05:34 dist-test-slave-tc2s krb5kdc[21069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028734, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:05:34.452508 21230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:34.445221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:42807 (local address 127.0.236.126:39761)
0806 15:05:34.445291 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:34.445293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:34.445409 (+ 116us) server_negotiation.cc:407] Connection header received
0806 15:05:34.445814 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:34.445815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:34.445825 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:34.445871 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:34.446733 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.447216 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.448264 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.448549 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.448606 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:34.450519 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:34.450520 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:34.450521 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:34.450527 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:34.451547 (+ 1020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:34.451867 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:34.451867 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:34.451867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:34.451909 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:34.452071 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:34.452072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:34.452072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:34.452161 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:34.452204 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:34.452387 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 15:05:34.452446 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 15:05:34.452642 21853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:34.445180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.67:42807)
0806 15:05:34.445311 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:34.445341 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:34.445745 (+ 404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:34.445977 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:34.445980 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:34.446493 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:34.446662 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:34.446666 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.447274 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.447275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:34.448163 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:34.448166 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.448627 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.448628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:34.448767 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:34.449151 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:34.449158 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:34.450444 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:34.451608 (+ 1164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:34.451614 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:34.451616 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:34.451811 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:34.451949 (+ 138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:34.451949 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:34.451949 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:34.452030 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:34.452202 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:34.452205 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:34.452299 (+ 94us) client_negotiation.cc:769] Sending connection context
0806 15:05:34.452369 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 15:05:34.452481 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":73,"threads_started":1}
I20260806 15:05:34.453119 21851 heartbeater.cc:347] Connected to a master server at 127.0.236.126:39761
I20260806 15:05:34.453195 21851 heartbeater.cc:464] Registering TS with master...
I20260806 15:05:34.453351 21851 heartbeater.cc:511] Master 127.0.236.126:39761 requested a full tablet report, sending...
I20260806 15:05:34.453709 21139 ts_manager.cc:194] Registered new tserver with Master: 925546e3fb0c4e378b64d8715107c0ef (127.0.236.67:46099)
I20260806 15:05:34.454092 21139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.67@KRBTEST.COM'} at 127.0.236.67:42807
I20260806 15:05:34.459365 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:05:34.465391 21230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:34.460613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44558 (local address 127.0.236.126:39761)
0806 15:05:34.460656 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:34.460657 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:34.460740 (+ 83us) server_negotiation.cc:407] Connection header received
0806 15:05:34.460790 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:34.460791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:34.460798 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:34.460830 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:34.461296 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.461726 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.462403 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.462575 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.462616 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:34.463522 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:34.463523 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:34.463523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:34.463529 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:34.464650 (+ 1121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:34.464882 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:34.464882 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:34.464882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:34.464928 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:34.465041 (+ 113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:34.465041 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:34.465041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:34.465151 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:34.465215 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:34.465287 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 15:05:34.465337 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 15:05:34.466657 21139 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44558:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:05:34.468089 21139 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:05:34.473693 21872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:34.470188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51614 (local address 127.0.236.67:46099)
0806 15:05:34.470364 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:34.470367 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:34.470374 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:05:34.470399 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:34.470401 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:34.470420 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:34.470482 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:05:34.470648 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.471426 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.472707 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.473332 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.473391 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:34.473444 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:34.473590 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 15:05:34.473626 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":70,"threads_started":1}
I20260806 15:05:34.473691 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:34.469837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:46099 (local address 127.0.0.1:51614)
0806 15:05:34.469969 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:34.470006 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:34.470067 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:34.470502 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:34.470505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:34.470507 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:05:34.470565 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:34.470567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.471526 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.471527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:34.472617 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:34.472620 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.473408 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.473409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:34.473500 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:34.473512 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 15:05:34.473570 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 15:05:34.473621 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":83,"threads_started":1}
I20260806 15:05:34.474090 21867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:34.470071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:34101 (local address 127.0.0.1:54378)
0806 15:05:34.470193 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:34.470203 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:34.470253 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:34.470814 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:34.470814 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:34.470816 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:05:34.470848 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:34.470849 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.472015 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.472016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:34.472820 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:34.472822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.473706 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.473707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:34.473920 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:34.473927 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 15:05:34.473984 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 15:05:34.474023 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
I20260806 15:05:34.474123 21873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:34.470171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54378 (local address 127.0.236.66:34101)
0806 15:05:34.470601 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:34.470605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:34.470615 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:05:34.470651 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:34.470653 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:34.470668 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:34.470734 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:05:34.470919 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.471942 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.472896 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.473553 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.473619 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:34.473668 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:34.474006 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 15:05:34.474049 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":93,"threads_started":1}
I20260806 15:05:34.474368 21737 tablet_service.cc:1467] Processing CreateTablet for tablet 87216c519f5c4dd9bfc243e5d9ecd4c0 (DEFAULT_TABLE table=test-table [id=56cda9aa127e415db84816adbaa8b2d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:05:34.474530 21870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:34.469999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45636 (local address 127.0.236.65:34465)
0806 15:05:34.470361 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:34.470366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:34.470375 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:05:34.470407 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:34.470408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:34.470420 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:34.470479 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:05:34.470753 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.471848 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.473106 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.473998 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.474053 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:34.474107 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:34.474402 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 15:05:34.474456 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":89,"threads_started":1}
I20260806 15:05:34.474699 21737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87216c519f5c4dd9bfc243e5d9ecd4c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:05:34.474869 21532 tablet_service.cc:1467] Processing CreateTablet for tablet 87216c519f5c4dd9bfc243e5d9ecd4c0 (DEFAULT_TABLE table=test-table [id=56cda9aa127e415db84816adbaa8b2d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:05:34.475209 21532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87216c519f5c4dd9bfc243e5d9ecd4c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:05:34.477170 21874 tablet_bootstrap.cc:492] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 925546e3fb0c4e378b64d8715107c0ef: Bootstrap starting.
I20260806 15:05:34.474478 21871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:34.469865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:34465 (local address 127.0.0.1:45636)
0806 15:05:34.470176 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:34.470185 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:34.470224 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:34.470644 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:34.470645 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:34.470646 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:05:34.470674 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:34.470675 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.471953 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.471954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:34.473027 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:34.473030 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.474142 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.474143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:34.474318 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:34.474321 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:05:34.474378 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 15:05:34.474415 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":22,"thread_start_us":113,"threads_started":1}
I20260806 15:05:34.477496 21876 tablet_bootstrap.cc:492] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 0be9946ecc994c248f9ac8a34da06763: Bootstrap starting.
I20260806 15:05:34.477836 21874 tablet_bootstrap.cc:654] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 925546e3fb0c4e378b64d8715107c0ef: Neither blocks nor log segments found. Creating new log.
I20260806 15:05:34.478211 21327 tablet_service.cc:1467] Processing CreateTablet for tablet 87216c519f5c4dd9bfc243e5d9ecd4c0 (DEFAULT_TABLE table=test-table [id=56cda9aa127e415db84816adbaa8b2d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:05:34.478375 21876 tablet_bootstrap.cc:654] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 0be9946ecc994c248f9ac8a34da06763: Neither blocks nor log segments found. Creating new log.
I20260806 15:05:34.478374 21874 log.cc:824] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 925546e3fb0c4e378b64d8715107c0ef: Log is configured to *not* fsync() on all Append() calls
I20260806 15:05:34.478542 21327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87216c519f5c4dd9bfc243e5d9ecd4c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:05:34.478909 21876 log.cc:824] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 0be9946ecc994c248f9ac8a34da06763: Log is configured to *not* fsync() on all Append() calls
I20260806 15:05:34.479096 21874 tablet_bootstrap.cc:492] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 925546e3fb0c4e378b64d8715107c0ef: No bootstrap required, opened a new log
I20260806 15:05:34.479159 21874 ts_tablet_manager.cc:1397] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 925546e3fb0c4e378b64d8715107c0ef: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:05:34.479750 21876 tablet_bootstrap.cc:492] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 0be9946ecc994c248f9ac8a34da06763: No bootstrap required, opened a new log
I20260806 15:05:34.479825 21876 ts_tablet_manager.cc:1397] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 0be9946ecc994c248f9ac8a34da06763: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:05:34.480589 21874 raft_consensus.cc:348] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 925546e3fb0c4e378b64d8715107c0ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:34.480723 21874 raft_consensus.cc:374] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 925546e3fb0c4e378b64d8715107c0ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:05:34.480756 21874 raft_consensus.cc:729] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 925546e3fb0c4e378b64d8715107c0ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 925546e3fb0c4e378b64d8715107c0ef, State: Initialized, Role: FOLLOWER
I20260806 15:05:34.480899 21874 consensus_queue.cc:260] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 925546e3fb0c4e378b64d8715107c0ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:34.481150 21874 ts_tablet_manager.cc:1428] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 925546e3fb0c4e378b64d8715107c0ef: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:05:34.481271 21851 heartbeater.cc:503] Master 127.0.236.126:39761 was elected leader, sending a full tablet report...
I20260806 15:05:34.481599 21881 tablet_bootstrap.cc:492] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f: Bootstrap starting.
I20260806 15:05:34.482003 21876 raft_consensus.cc:348] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 0be9946ecc994c248f9ac8a34da06763 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:34.482170 21876 raft_consensus.cc:374] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 0be9946ecc994c248f9ac8a34da06763 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:05:34.482200 21876 raft_consensus.cc:729] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 0be9946ecc994c248f9ac8a34da06763 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0be9946ecc994c248f9ac8a34da06763, State: Initialized, Role: FOLLOWER
I20260806 15:05:34.482254 21881 tablet_bootstrap.cc:654] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f: Neither blocks nor log segments found. Creating new log.
I20260806 15:05:34.482304 21876 consensus_queue.cc:260] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 0be9946ecc994c248f9ac8a34da06763 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:34.482561 21633 heartbeater.cc:503] Master 127.0.236.126:39761 was elected leader, sending a full tablet report...
I20260806 15:05:34.482563 21876 ts_tablet_manager.cc:1428] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 0be9946ecc994c248f9ac8a34da06763: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 15:05:34.482788 21881 log.cc:824] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f: Log is configured to *not* fsync() on all Append() calls
I20260806 15:05:34.483508 21881 tablet_bootstrap.cc:492] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f: No bootstrap required, opened a new log
I20260806 15:05:34.483570 21881 ts_tablet_manager.cc:1397] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 15:05:34.484959 21881 raft_consensus.cc:348] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:34.485082 21881 raft_consensus.cc:374] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:05:34.485116 21881 raft_consensus.cc:729] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e08410ceb93456ea29bb3eb223bd15f, State: Initialized, Role: FOLLOWER
I20260806 15:05:34.485219 21881 consensus_queue.cc:260] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:34.485466 21881 ts_tablet_manager.cc:1428] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 15:05:34.485493 21430 heartbeater.cc:503] Master 127.0.236.126:39761 was elected leader, sending a full tablet report...
W20260806 15:05:34.551368 21636 tablet.cc:2369] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 0be9946ecc994c248f9ac8a34da06763: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 15:05:34.661123 21431 tablet.cc:2369] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 15:05:34.695107 21852 tablet.cc:2369] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 925546e3fb0c4e378b64d8715107c0ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:05:34.740980 21885 raft_consensus.cc:482] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:05:34.741098 21885 raft_consensus.cc:504] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:34.741430 21885 leader_election.cc:288] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 925546e3fb0c4e378b64d8715107c0ef (127.0.236.67:46099), 0be9946ecc994c248f9ac8a34da06763 (127.0.236.66:34101)
I20260806 15:05:34.745369 21873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:34.741953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:34191 (local address 127.0.236.66:34101)
0806 15:05:34.741998 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:34.742000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:34.742009 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:05:34.742053 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:34.742054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:34.742062 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:34.742110 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:05:34.742256 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.743005 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.744442 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.745025 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.745093 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:34.745102 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:34.745262 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 15:05:34.745298 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 15:05:34.745369 21872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:34.741756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:54983 (local address 127.0.236.67:46099)
0806 15:05:34.741819 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:34.741821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:34.741878 (+ 57us) server_negotiation.cc:407] Connection header received
0806 15:05:34.742001 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:34.742002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:34.742013 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:34.742065 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:05:34.742232 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.742988 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.744442 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.745025 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.745095 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:34.745102 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:34.745262 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 15:05:34.745298 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 15:05:34.745410 21886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:34.741670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:46099 (local address 127.0.236.65:54983)
0806 15:05:34.741791 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:34.741805 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:34.741912 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:34.742079 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:34.742079 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:34.742081 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:05:34.742162 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:34.742164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.743062 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.743063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:34.744354 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:34.744357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.745102 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.745104 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:34.745197 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:34.745203 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 15:05:34.745251 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 15:05:34.745300 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":34,"thread_start_us":63,"threads_started":1}
I20260806 15:05:34.745576 21887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:34.741842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:34101 (local address 127.0.236.65:34191)
0806 15:05:34.741956 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:34.741965 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:34.742008 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:34.742128 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:34.742129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:34.742130 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:05:34.742195 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:34.742196 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.743071 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.743072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:34.744358 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:34.744360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.745104 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.745105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:34.745197 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:34.745202 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 15:05:34.745251 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 15:05:34.745300 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":67,"thread_start_us":41,"threads_started":1}
I20260806 15:05:34.745725 21769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87216c519f5c4dd9bfc243e5d9ecd4c0" candidate_uuid: "5e08410ceb93456ea29bb3eb223bd15f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "925546e3fb0c4e378b64d8715107c0ef" is_pre_election: true
I20260806 15:05:34.745874 21769 raft_consensus.cc:2454] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 925546e3fb0c4e378b64d8715107c0ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e08410ceb93456ea29bb3eb223bd15f in term 0.
I20260806 15:05:34.746136 21258 leader_election.cc:302] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e08410ceb93456ea29bb3eb223bd15f, 925546e3fb0c4e378b64d8715107c0ef; no voters:
I20260806 15:05:34.746239 21564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87216c519f5c4dd9bfc243e5d9ecd4c0" candidate_uuid: "5e08410ceb93456ea29bb3eb223bd15f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0be9946ecc994c248f9ac8a34da06763" is_pre_election: true
I20260806 15:05:34.746297 21885 raft_consensus.cc:2781] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:05:34.746359 21885 raft_consensus.cc:482] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:05:34.746390 21885 raft_consensus.cc:3037] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:05:34.746407 21564 raft_consensus.cc:2454] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 0be9946ecc994c248f9ac8a34da06763 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e08410ceb93456ea29bb3eb223bd15f in term 0.
I20260806 15:05:34.747114 21885 raft_consensus.cc:504] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:34.747223 21885 leader_election.cc:288] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 1 election: Requested vote from peers 925546e3fb0c4e378b64d8715107c0ef (127.0.236.67:46099), 0be9946ecc994c248f9ac8a34da06763 (127.0.236.66:34101)
I20260806 15:05:34.747437 21564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87216c519f5c4dd9bfc243e5d9ecd4c0" candidate_uuid: "5e08410ceb93456ea29bb3eb223bd15f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0be9946ecc994c248f9ac8a34da06763"
I20260806 15:05:34.747455 21769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87216c519f5c4dd9bfc243e5d9ecd4c0" candidate_uuid: "5e08410ceb93456ea29bb3eb223bd15f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "925546e3fb0c4e378b64d8715107c0ef"
I20260806 15:05:34.747529 21564 raft_consensus.cc:3037] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 0be9946ecc994c248f9ac8a34da06763 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:05:34.747529 21769 raft_consensus.cc:3037] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 925546e3fb0c4e378b64d8715107c0ef [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:05:34.748387 21564 raft_consensus.cc:2454] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 0be9946ecc994c248f9ac8a34da06763 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e08410ceb93456ea29bb3eb223bd15f in term 1.
I20260806 15:05:34.748407 21769 raft_consensus.cc:2454] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 925546e3fb0c4e378b64d8715107c0ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e08410ceb93456ea29bb3eb223bd15f in term 1.
I20260806 15:05:34.748558 21258 leader_election.cc:302] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0be9946ecc994c248f9ac8a34da06763, 5e08410ceb93456ea29bb3eb223bd15f; no voters:
I20260806 15:05:34.748641 21885 raft_consensus.cc:2781] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:05:34.748785 21885 raft_consensus.cc:686] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 LEADER]: Becoming Leader. State: Replica: 5e08410ceb93456ea29bb3eb223bd15f, State: Running, Role: LEADER
I20260806 15:05:34.748955 21885 consensus_queue.cc:237] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:34.749794 21139 catalog_manager.cc:5495] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e08410ceb93456ea29bb3eb223bd15f (127.0.236.65). New cstate: current_term: 1 leader_uuid: "5e08410ceb93456ea29bb3eb223bd15f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } health_report { overall_health: HEALTHY } } }
I20260806 15:05:34.822530 21870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:34.820524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45644 (local address 127.0.236.65:34465)
0806 15:05:34.820577 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:34.820579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:34.820599 (+ 20us) server_negotiation.cc:407] Connection header received
0806 15:05:34.820633 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:34.820634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:34.820652 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:34.820701 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:05:34.820800 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.821254 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.821950 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:34.822125 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:34.822167 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:34.822277 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:05:34.822372 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:05:34.822426 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:34.822442 (+ 16us) server_negotiation.cc:299] Negotiation successful
0806 15:05:34.822459 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 15:05:34.824234 21769 raft_consensus.cc:1261] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 925546e3fb0c4e378b64d8715107c0ef [term 1 FOLLOWER]: Refusing update from remote peer 5e08410ceb93456ea29bb3eb223bd15f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:05:34.824240 21564 raft_consensus.cc:1261] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 0be9946ecc994c248f9ac8a34da06763 [term 1 FOLLOWER]: Refusing update from remote peer 5e08410ceb93456ea29bb3eb223bd15f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:05:34.824581 21889 consensus_queue.cc:1035] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f [LEADER]: Connected to new peer: Peer: permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:05:34.824687 21885 consensus_queue.cc:1035] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:05:34.829340 21137 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:05:34.829424 21137 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:05:34.830123 21137 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f: Sending DeleteTablet for 3 replicas of tablet 87216c519f5c4dd9bfc243e5d9ecd4c0
I20260806 15:05:34.830430 21532 tablet_service.cc:1514] Processing DeleteTablet for tablet 87216c519f5c4dd9bfc243e5d9ecd4c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:05:34 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:54378
I20260806 15:05:34.830451 21737 tablet_service.cc:1514] Processing DeleteTablet for tablet 87216c519f5c4dd9bfc243e5d9ecd4c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:05:34 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:51614
I20260806 15:05:34.830451 21327 tablet_service.cc:1514] Processing DeleteTablet for tablet 87216c519f5c4dd9bfc243e5d9ecd4c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:05:34 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:45636
I20260806 15:05:34.830698 21909 tablet_replica.cc:330] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 0be9946ecc994c248f9ac8a34da06763: stopping tablet replica
I20260806 15:05:34.830718 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 21236
I20260806 15:05:34.830821 21908 tablet_replica.cc:330] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 925546e3fb0c4e378b64d8715107c0ef: stopping tablet replica
I20260806 15:05:34.830974 21908 raft_consensus.cc:2229] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 925546e3fb0c4e378b64d8715107c0ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:05:34.831106 21909 raft_consensus.cc:2229] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 0be9946ecc994c248f9ac8a34da06763 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:05:34.831297 21909 raft_consensus.cc:2258] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 0be9946ecc994c248f9ac8a34da06763 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:05:34.831496 21908 raft_consensus.cc:2258] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 925546e3fb0c4e378b64d8715107c0ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:05:34.833528 21909 ts_tablet_manager.cc:1905] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 0be9946ecc994c248f9ac8a34da06763: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:05:34.835944 21909 ts_tablet_manager.cc:1918] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 0be9946ecc994c248f9ac8a34da06763: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:05:34.836014 21909 log.cc:1196] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 0be9946ecc994c248f9ac8a34da06763: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/wal/wals/87216c519f5c4dd9bfc243e5d9ecd4c0
I20260806 15:05:34.836172 21908 ts_tablet_manager.cc:1905] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 925546e3fb0c4e378b64d8715107c0ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:05:34.836251 21909 ts_tablet_manager.cc:1939] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 0be9946ecc994c248f9ac8a34da06763: Deleting consensus metadata
I20260806 15:05:34.836634 21118 catalog_manager.cc:4841] TS 0be9946ecc994c248f9ac8a34da06763 (127.0.236.66:34101): tablet 87216c519f5c4dd9bfc243e5d9ecd4c0 (table test-table [id=56cda9aa127e415db84816adbaa8b2d8]) successfully deleted
I20260806 15:05:34.838100 21908 ts_tablet_manager.cc:1918] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 925546e3fb0c4e378b64d8715107c0ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:05:34.838203 21908 log.cc:1196] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 925546e3fb0c4e378b64d8715107c0ef: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/87216c519f5c4dd9bfc243e5d9ecd4c0
I20260806 15:05:34.838498 21908 ts_tablet_manager.cc:1939] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 925546e3fb0c4e378b64d8715107c0ef: Deleting consensus metadata
I20260806 15:05:34.838871 21118 catalog_manager.cc:4841] TS 925546e3fb0c4e378b64d8715107c0ef (127.0.236.67:46099): tablet 87216c519f5c4dd9bfc243e5d9ecd4c0 (table test-table [id=56cda9aa127e415db84816adbaa8b2d8]) successfully deleted
W20260806 15:05:34.846621 21121 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.236.65:34465 (error 108)
I20260806 15:05:34.846815 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 21437
I20260806 15:05:34.849547 21871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:34.846967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:34465 (local address 127.0.0.1:45646)
0806 15:05:34.849454 (+ 2487us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:34.849508 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.65:34465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2464}
W20260806 15:05:34.849699 21121 catalog_manager.cc:4584] TS 5e08410ceb93456ea29bb3eb223bd15f (127.0.236.65:34465): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 87216c519f5c4dd9bfc243e5d9ecd4c0: Network error: Client connection negotiation failed: client connection to 127.0.236.65:34465: connect: Connection refused (error 111)
I20260806 15:05:34.856252 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 21641
I20260806 15:05:34.864873 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 21087
I20260806 15:05:34.876307 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:39761
--webserver_interface=127.0.236.126
--webserver_port=40577
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:35035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:39761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:05:34.955542 21912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:34.955726 21912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:34.955746 21912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:34.957185 21912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:05:34.957227 21912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:34.957240 21912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:34.957252 21912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:05:34.957262 21912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:05:34.958735 21912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:35035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:39761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:39761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=40577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:34.958954 21912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:34.959146 21912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:05:34.968997 21920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:34.969043 21912 server_base.cc:1034] running on GCE node
W20260806 15:05:34.968992 21919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:34.968993 21922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:34.969455 21912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:34.969751 21912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:34.970887 21912 hybrid_clock.cc:648] HybridClock initialized: now 1786028734970855 us; error 44 us; skew 500 ppm
Aug 06 15:05:34 dist-test-slave-tc2s krb5kdc[21069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028734, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:34.979456 21912 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:05:34.979861 21912 webserver.cc:460] Webserver started at http://127.0.236.126:40577/ using document root <none> and password file <none>
I20260806 15:05:34.980046 21912 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:34.980096 21912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:05:34.981299 21912 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:05:34.981868 21932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:34.982023 21912 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:34.982105 21912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "e58f8c49b33846ea9e314f0723da7a9f"
format_stamp: "Formatted at 2026-08-06 15:05:33 on dist-test-slave-tc2s"
server_key: "5d9a00fe1a4ab734dae8286e929fbb7f"
server_key_iv: "43e6c129b190aab570e4fb18c8076d55"
server_key_version: "encryptionkey@0"
I20260806 15:05:34.982450 21912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:35.006677 21912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:35.008239 21912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:35.008358 21912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:35.012181 21912 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:39761
I20260806 15:05:35.012216 22023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:39761 every 8 connection(s)
I20260806 15:05:35.012550 21912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:05:35.014146 22025 sys_catalog.cc:263] Verifying existing consensus state
I20260806 15:05:35.014748 22025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f: Bootstrap starting.
I20260806 15:05:35.020972 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 21912
I20260806 15:05:35.021303 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:34465
--local_ip_for_outbound_sockets=127.0.236.65
--tserver_master_addrs=127.0.236.126:39761
--webserver_port=37915
--webserver_interface=127.0.236.65
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:35035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:05:35.023336 22025 log.cc:824] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f: Log is configured to *not* fsync() on all Append() calls
I20260806 15:05:35.025867 22025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 15:05:35.026175 22025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f: Bootstrap complete.
I20260806 15:05:35.029247 22025 raft_consensus.cc:348] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e58f8c49b33846ea9e314f0723da7a9f" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39761 } }
I20260806 15:05:35.029561 22025 raft_consensus.cc:729] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e58f8c49b33846ea9e314f0723da7a9f, State: Initialized, Role: FOLLOWER
I20260806 15:05:35.029732 22025 consensus_queue.cc:260] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e58f8c49b33846ea9e314f0723da7a9f" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39761 } }
I20260806 15:05:35.029805 22025 raft_consensus.cc:388] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:05:35.029858 22025 raft_consensus.cc:482] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:05:35.029928 22025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [term 1 FOLLOWER]: Advancing to term 2
I20260806 15:05:35.031246 22025 raft_consensus.cc:504] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e58f8c49b33846ea9e314f0723da7a9f" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39761 } }
I20260806 15:05:35.031359 22025 leader_election.cc:302] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e58f8c49b33846ea9e314f0723da7a9f; no voters:
I20260806 15:05:35.031534 22025 leader_election.cc:288] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 15:05:35.031565 22031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [term 2 FOLLOWER]: Leader election won for term 2
I20260806 15:05:35.031704 22031 raft_consensus.cc:686] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [term 2 LEADER]: Becoming Leader. State: Replica: e58f8c49b33846ea9e314f0723da7a9f, State: Running, Role: LEADER
I20260806 15:05:35.031785 22025 sys_catalog.cc:564] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:05:35.031798 22031 consensus_queue.cc:237] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e58f8c49b33846ea9e314f0723da7a9f" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39761 } }
I20260806 15:05:35.032109 22031 sys_catalog.cc:455] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e58f8c49b33846ea9e314f0723da7a9f. Latest consensus state: current_term: 2 leader_uuid: "e58f8c49b33846ea9e314f0723da7a9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e58f8c49b33846ea9e314f0723da7a9f" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39761 } } }
I20260806 15:05:35.032177 22031 sys_catalog.cc:458] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [sys.catalog]: This master's current role is: LEADER
I20260806 15:05:35.032289 22033 sys_catalog.cc:455] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e58f8c49b33846ea9e314f0723da7a9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e58f8c49b33846ea9e314f0723da7a9f" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39761 } } }
I20260806 15:05:35.032366 22033 sys_catalog.cc:458] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [sys.catalog]: This master's current role is: LEADER
I20260806 15:05:35.032874 22039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:05:35.033701 22039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:05:35.033803 22039 catalog_manager.cc:1259] Loaded cluster ID: 43da28a2a2de450ead475558254a73e2
I20260806 15:05:35.033833 22039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:05:35.034180 22039 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 15:05:35.038785 22039 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 15:05:35.041034 22039 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:05:35.041219 22039 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f: Loaded TSK: 0
I20260806 15:05:35.041456 22039 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 15:05:35.108649 22029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:35.108835 22029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:35.108865 22029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:35.109997 22029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:35.110041 22029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:35.110077 22029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:05:35.111485 22029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:35035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:34465
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=37915
--tserver_master_addrs=127.0.236.126:39761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:35.111698 22029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:35.111882 22029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:05:35.116369 22073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:35.116364 22065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:35.116600 22067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:35.116577 22029 server_base.cc:1034] running on GCE node
I20260806 15:05:35.116858 22029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:35.117136 22029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:35.118315 22029 hybrid_clock.cc:648] HybridClock initialized: now 1786028735118294 us; error 32 us; skew 500 ppm
Aug 06 15:05:35 dist-test-slave-tc2s krb5kdc[21069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028735, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:35.121266 22029 init.cc:377] Logged in from keytab as kudu/127.0.236.65@KRBTEST.COM (short username kudu)
I20260806 15:05:35.121802 22029 webserver.cc:460] Webserver started at http://127.0.236.65:37915/ using document root <none> and password file <none>
I20260806 15:05:35.122094 22029 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:35.122175 22029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:05:35.123795 22029 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:05:35.124492 22081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:35.124655 22029 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:05:35.124747 22029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "5e08410ceb93456ea29bb3eb223bd15f"
format_stamp: "Formatted at 2026-08-06 15:05:34 on dist-test-slave-tc2s"
server_key: "77e5c0221ea6f3457fe584f6278b8acf"
server_key_iv: "958e45693423899335221e41aeaa23db"
server_key_version: "encryptionkey@0"
I20260806 15:05:35.125104 22029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:35.142481 22029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:35.144584 22029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:35.144728 22029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:35.144958 22029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:05:35.145417 22094 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 15:05:35.146869 22029 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 15:05:35.146934 22029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 15:05:35.146965 22029 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 15:05:35.147789 22029 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 15:05:35.147835 22029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 15:05:35.147889 22094 tablet_bootstrap.cc:492] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f: Bootstrap starting.
I20260806 15:05:35.154604 22029 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:34465
I20260806 15:05:35.154981 22029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:05:35.157529 22094 log.cc:824] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f: Log is configured to *not* fsync() on all Append() calls
I20260806 15:05:35.158517 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 22029
I20260806 15:05:35.158813 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:34101
--local_ip_for_outbound_sockets=127.0.236.66
--tserver_master_addrs=127.0.236.126:39761
--webserver_port=33159
--webserver_interface=127.0.236.66
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:35035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:05:35.161343 22094 tablet_bootstrap.cc:492] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 15:05:35.161630 22094 tablet_bootstrap.cc:492] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f: Bootstrap complete.
I20260806 15:05:35.162828 22094 ts_tablet_manager.cc:1397] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.006s
I20260806 15:05:35.164199 22094 raft_consensus.cc:348] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:35.164513 22094 raft_consensus.cc:729] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e08410ceb93456ea29bb3eb223bd15f, State: Initialized, Role: FOLLOWER
I20260806 15:05:35.164656 22094 consensus_queue.cc:260] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:35.164917 22094 ts_tablet_manager.cc:1428] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 15:05:35.165578 22271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:34465 every 8 connection(s)
Aug 06 15:05:35 dist-test-slave-tc2s krb5kdc[21069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028735, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:05:35.171514 22275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:35.155809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.65:39469)
0806 15:05:35.157775 (+ 1966us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:35.157817 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:35.158322 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:35.158952 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:35.158956 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:35.159268 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:35.159538 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:35.159544 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.161131 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.161132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:35.162458 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:35.162462 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.164493 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.164494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:35.164859 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:35.165815 (+ 956us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:35.165824 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:35.167626 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:35.169707 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:35.169715 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:35.169718 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:35.170044 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:35.170316 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:35.170317 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:35.170318 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:35.170464 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:35.170870 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:35.170876 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:35.171035 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 15:05:35.171144 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 15:05:35.171305 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1929,"thread_start_us":91,"threads_started":1}
I20260806 15:05:35.171526 22277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:35.155905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:39469 (local address 127.0.236.126:39761)
0806 15:05:35.156117 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:35.156128 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:35.157951 (+ 1823us) server_negotiation.cc:407] Connection header received
0806 15:05:35.158560 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:35.158569 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:35.158632 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:35.158738 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:35.159672 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.161007 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.162573 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.163016 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.163075 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:35.167751 (+ 4676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:35.167755 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:35.167756 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:35.167768 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:35.169616 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:35.170138 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:35.170141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:35.170142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:35.170250 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:35.170536 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:35.170537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:35.170537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:35.170800 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:35.170874 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:35.171218 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 15:05:35.171348 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":96,"threads_started":1}
I20260806 15:05:35.172763 22272 heartbeater.cc:347] Connected to a master server at 127.0.236.126:39761
I20260806 15:05:35.172860 22272 heartbeater.cc:464] Registering TS with master...
I20260806 15:05:35.173030 22272 heartbeater.cc:511] Master 127.0.236.126:39761 requested a full tablet report, sending...
I20260806 15:05:35.173676 21955 ts_manager.cc:194] Registered new tserver with Master: 5e08410ceb93456ea29bb3eb223bd15f (127.0.236.65:34465)
I20260806 15:05:35.174714 21955 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.65@KRBTEST.COM'} at 127.0.236.65:39469
I20260806 15:05:35.175616 22272 heartbeater.cc:503] Master 127.0.236.126:39761 was elected leader, sending a full tablet report...
Aug 06 15:05:35 dist-test-slave-tc2s krb5kdc[21069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028734, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for kudu/127.0.236.65@KRBTEST.COM
I20260806 15:05:35.181458 22285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:35.174311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45654 (local address 127.0.236.65:34465)
0806 15:05:35.174497 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:35.174501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:35.174657 (+ 156us) server_negotiation.cc:407] Connection header received
0806 15:05:35.174727 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:35.174729 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:35.174744 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:35.174865 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:35.175343 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.176136 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.176831 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.177118 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.177175 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:35.179066 (+ 1891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:35.179074 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:35.179074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:35.179079 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:35.180404 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:35.180795 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:35.180797 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:35.180797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:35.180844 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:35.181009 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:35.181009 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:35.181010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:35.181122 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:35.181178 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:35.181330 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 15:05:35.181398 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":137,"thread_start_us":75,"threads_started":1}
I20260806 15:05:35.181561 22283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:35.174260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:34465 (local address 127.0.0.1:45654)
0806 15:05:35.174555 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:35.174586 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:35.174655 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:35.174921 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:35.174926 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:35.175227 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:35.175268 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:35.175270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.176210 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.176211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:35.176755 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:35.176757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.177186 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.177187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:35.177273 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:35.177670 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:35.177674 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:35.178989 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:35.180473 (+ 1484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:35.180480 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:35.180482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:35.180729 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:35.180894 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:35.180895 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:35.180895 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:35.180959 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:35.181172 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:35.181179 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:35.181229 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 15:05:35.181305 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 15:05:35.181418 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":90,"threads_started":1}
I20260806 15:05:35.181879 22129 tablet_service.cc:1514] Processing DeleteTablet for tablet 87216c519f5c4dd9bfc243e5d9ecd4c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:05:34 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:45654
I20260806 15:05:35.182348 22287 tablet_replica.cc:330] stopping tablet replica
I20260806 15:05:35.182430 22287 raft_consensus.cc:2229] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:05:35.182498 22287 raft_consensus.cc:2258] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:05:35.182744 22287 ts_tablet_manager.cc:1905] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:05:35.184626 22287 ts_tablet_manager.cc:1918] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:05:35.184707 22287 log.cc:1196] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/wal/wals/87216c519f5c4dd9bfc243e5d9ecd4c0
I20260806 15:05:35.184927 22287 ts_tablet_manager.cc:1939] T 87216c519f5c4dd9bfc243e5d9ecd4c0 P 5e08410ceb93456ea29bb3eb223bd15f: Deleting consensus metadata
I20260806 15:05:35.185359 21936 catalog_manager.cc:4841] TS 5e08410ceb93456ea29bb3eb223bd15f (127.0.236.65:34465): tablet 87216c519f5c4dd9bfc243e5d9ecd4c0 (table test-table [id=56cda9aa127e415db84816adbaa8b2d8]) successfully deleted
W20260806 15:05:35.244275 22280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:35.244457 22280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:35.244489 22280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:35.245582 22280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:35.245620 22280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:35.245654 22280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:05:35.247062 22280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:35035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:34101
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=33159
--tserver_master_addrs=127.0.236.126:39761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:35.247282 22280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:35.247479 22280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:05:35.252067 22295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:35.252085 22280 server_base.cc:1034] running on GCE node
W20260806 15:05:35.252106 22294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:35.252070 22297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:35.252465 22280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:35.252679 22280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:35.253814 22280 hybrid_clock.cc:648] HybridClock initialized: now 1786028735253802 us; error 24 us; skew 500 ppm
Aug 06 15:05:35 dist-test-slave-tc2s krb5kdc[21069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028735, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:35.256129 22280 init.cc:377] Logged in from keytab as kudu/127.0.236.66@KRBTEST.COM (short username kudu)
I20260806 15:05:35.256505 22280 webserver.cc:460] Webserver started at http://127.0.236.66:33159/ using document root <none> and password file <none>
I20260806 15:05:35.256705 22280 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:35.256764 22280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:05:35.257992 22280 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:05:35.258498 22305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:35.258636 22280 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:05:35.258716 22280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "0be9946ecc994c248f9ac8a34da06763"
format_stamp: "Formatted at 2026-08-06 15:05:34 on dist-test-slave-tc2s"
server_key: "bc6eee00b36ea72367bd00dfdd03aeff"
server_key_iv: "7a5c48eb9584ca18f4053cd9878ab378"
server_key_version: "encryptionkey@0"
I20260806 15:05:35.259002 22280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:35.270780 22280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:35.272317 22280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:35.272436 22280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:35.272637 22280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:05:35.272918 22280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:05:35.272956 22280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:35.272979 22280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:05:35.273000 22280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:35.278461 22280 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:34101
I20260806 15:05:35.278515 22479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:34101 every 8 connection(s)
I20260806 15:05:35.278785 22280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
Aug 06 15:05:35 dist-test-slave-tc2s krb5kdc[21069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028735, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:05:35.285709 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 22280
I20260806 15:05:35.286052 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:46099
--local_ip_for_outbound_sockets=127.0.236.67
--tserver_master_addrs=127.0.236.126:39761
--webserver_port=37337
--webserver_interface=127.0.236.67
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:35035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:05:35.286989 22277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:35.279371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:44301 (local address 127.0.236.126:39761)
0806 15:05:35.279434 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:35.279435 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:35.279621 (+ 186us) server_negotiation.cc:407] Connection header received
0806 15:05:35.280053 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:35.280055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:35.280070 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:35.280119 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:35.280987 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.281472 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.282424 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.282632 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.282686 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:35.284526 (+ 1840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:35.284527 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:35.284527 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:35.284534 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:35.285582 (+ 1048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:35.286048 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:35.286049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:35.286049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:35.286102 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:35.286436 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:35.286437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:35.286437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:35.286534 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:35.286580 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:35.286858 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 15:05:35.286927 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 15:05:35.287190 22484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:35.279360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.66:44301)
0806 15:05:35.279519 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:35.279551 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:35.279966 (+ 415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:35.280177 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:35.280180 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:35.280718 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:35.280887 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:35.280891 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.281550 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.281551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:35.282342 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:35.282344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.282698 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.282699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:35.282807 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:35.283166 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:35.283173 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:35.284432 (+ 1259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:35.285649 (+ 1217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:35.285657 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:35.285658 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:35.285975 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:35.286248 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:35.286249 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:35.286250 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:35.286377 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:35.286586 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:35.286592 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:35.286744 (+ 152us) client_negotiation.cc:769] Sending connection context
0806 15:05:35.286849 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 15:05:35.286994 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":90,"threads_started":1}
I20260806 15:05:35.287662 22482 heartbeater.cc:347] Connected to a master server at 127.0.236.126:39761
I20260806 15:05:35.287750 22482 heartbeater.cc:464] Registering TS with master...
I20260806 15:05:35.287910 22482 heartbeater.cc:511] Master 127.0.236.126:39761 requested a full tablet report, sending...
I20260806 15:05:35.288270 21955 ts_manager.cc:194] Registered new tserver with Master: 0be9946ecc994c248f9ac8a34da06763 (127.0.236.66:34101)
I20260806 15:05:35.288611 21955 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.66@KRBTEST.COM'} at 127.0.236.66:44301
W20260806 15:05:35.365644 22485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:35.365844 22485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:35.365876 22485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:35.367012 22485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:35.367051 22485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:35.367076 22485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:05:35.368445 22485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:35035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:46099
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=37337
--tserver_master_addrs=127.0.236.126:39761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:35.368621 22485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:35.368782 22485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:05:35.373158 22490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:35.373168 22491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:35.373167 22495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:35.373344 22485 server_base.cc:1034] running on GCE node
I20260806 15:05:35.373504 22485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:35.373719 22485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:35.374851 22485 hybrid_clock.cc:648] HybridClock initialized: now 1786028735374838 us; error 27 us; skew 500 ppm
Aug 06 15:05:35 dist-test-slave-tc2s krb5kdc[21069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028735, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:35.377151 22485 init.cc:377] Logged in from keytab as kudu/127.0.236.67@KRBTEST.COM (short username kudu)
I20260806 15:05:35.377537 22485 webserver.cc:460] Webserver started at http://127.0.236.67:37337/ using document root <none> and password file <none>
I20260806 15:05:35.377715 22485 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:35.377777 22485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:05:35.378988 22485 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:05:35.379458 22503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:35.379737 22485 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 15:05:35.379815 22485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "925546e3fb0c4e378b64d8715107c0ef"
format_stamp: "Formatted at 2026-08-06 15:05:34 on dist-test-slave-tc2s"
server_key: "98516631d3b8cc386d30c57e1567eee6"
server_key_iv: "e2b373d9074aecd6cbccff6bf3f5c53e"
server_key_version: "encryptionkey@0"
I20260806 15:05:35.380086 22485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:35.401925 22485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:35.403470 22485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:35.403570 22485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:35.403738 22485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:05:35.403960 22485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:05:35.403987 22485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:35.404007 22485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:05:35.404023 22485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:35.410128 22485 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:46099
I20260806 15:05:35.410238 22683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:46099 every 8 connection(s)
I20260806 15:05:35.410542 22485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
I20260806 15:05:35.411692 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 22485
Aug 06 15:05:35 dist-test-slave-tc2s krb5kdc[21069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028735, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:05:35.419246 22689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:35.412751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44562 (local address 127.0.236.126:39761)
0806 15:05:35.412935 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:35.412936 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:35.412945 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:05:35.412978 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:35.412979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:35.412990 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:35.413054 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:35.413775 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.414514 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.415383 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.415604 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.415664 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:35.416770 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:35.416771 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:35.416772 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:35.416779 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:35.418178 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:35.418478 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:35.418479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:35.418479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:35.418538 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:35.418697 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:35.418698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:35.418698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:35.418804 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:35.418962 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:35.419000 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 15:05:35.419153 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":64,"threads_started":1}
I20260806 15:05:35.420461 22277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:35.411563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:58687 (local address 127.0.236.126:39761)
0806 15:05:35.411633 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:35.411635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:35.411646 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:05:35.412145 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:35.412146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:35.412159 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:35.412189 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:35.413431 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.414129 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.415523 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.415779 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.415846 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:35.417868 (+ 2022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:35.417869 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:35.417869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:35.417875 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:35.419193 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:35.419665 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:35.419666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:35.419666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:35.419725 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:35.419957 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:35.419957 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:35.419958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:35.420068 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:35.420129 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:35.420340 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 15:05:35.420398 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 15:05:35.420603 22687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:35.411238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.67:58687)
0806 15:05:35.411405 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:35.411447 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:35.412022 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:35.412355 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:35.412359 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:35.413074 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:35.413336 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:35.413342 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.414236 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.414237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:35.415442 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:35.415445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.415855 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.415856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:35.416006 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:35.416399 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:35.416408 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:35.417781 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:35.419277 (+ 1496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:35.419284 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:35.419286 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:35.419598 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:35.419779 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:35.419780 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:35.419781 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:35.419901 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:35.420121 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:35.420126 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:35.420236 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 15:05:35.420310 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 15:05:35.420432 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":87,"threads_started":1}
I20260806 15:05:35.420969 22684 heartbeater.cc:347] Connected to a master server at 127.0.236.126:39761
I20260806 15:05:35.421053 22684 heartbeater.cc:464] Registering TS with master...
I20260806 15:05:35.421213 22684 heartbeater.cc:511] Master 127.0.236.126:39761 requested a full tablet report, sending...
I20260806 15:05:35.421523 21955 ts_manager.cc:194] Registered new tserver with Master: 925546e3fb0c4e378b64d8715107c0ef (127.0.236.67:46099)
I20260806 15:05:35.421831 21955 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.67@KRBTEST.COM'} at 127.0.236.67:58687
I20260806 15:05:35.430655 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:05:35.432987 22277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:35.430919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44576 (local address 127.0.236.126:39761)
0806 15:05:35.430982 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:35.430984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:35.431024 (+ 40us) server_negotiation.cc:407] Connection header received
0806 15:05:35.431080 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:35.431080 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:35.431110 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:35.431160 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:05:35.431258 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.431683 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.432405 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.432582 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.432624 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:35.432726 (+ 102us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:05:35.432822 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:05:35.432891 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:35.432906 (+ 15us) server_negotiation.cc:299] Negotiation successful
0806 15:05:35.432920 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 15:05:35.433205 21955 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:44576:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:05:35.434770 21955 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:05:35.436682 22129 tablet_service.cc:1467] Processing CreateTablet for tablet a611aae3f2774ab88698e46ad221e6e5 (DEFAULT_TABLE table=test-table [id=11540116d3414fda89171f2fc00fb954]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:05:35.436862 22129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a611aae3f2774ab88698e46ad221e6e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:05:35.438159 22094 tablet_bootstrap.cc:492] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f: Bootstrap starting.
I20260806 15:05:35.438488 22094 tablet_bootstrap.cc:654] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f: Neither blocks nor log segments found. Creating new log.
I20260806 15:05:35.439046 22094 tablet_bootstrap.cc:492] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f: No bootstrap required, opened a new log
I20260806 15:05:35.439093 22094 ts_tablet_manager.cc:1397] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260806 15:05:35.439414 22094 raft_consensus.cc:348] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:35.439491 22094 raft_consensus.cc:374] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:05:35.439540 22094 raft_consensus.cc:729] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e08410ceb93456ea29bb3eb223bd15f, State: Initialized, Role: FOLLOWER
I20260806 15:05:35.439594 22094 consensus_queue.cc:260] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:35.439692 22094 ts_tablet_manager.cc:1428] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 15:05:35.440398 22283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:35.436366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:46099 (local address 127.0.0.1:51628)
0806 15:05:35.436489 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:35.436499 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:35.436538 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:35.437008 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:35.437008 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:35.437011 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:05:35.437075 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:35.437077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.438481 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.438482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:35.439300 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:35.439303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.440140 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.440141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:35.440211 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:35.440235 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 15:05:35.440283 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 15:05:35.440338 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":12}
I20260806 15:05:35.440613 22691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:35.436435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51628 (local address 127.0.236.67:46099)
0806 15:05:35.436843 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:35.436846 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:35.436855 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:05:35.436883 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:35.436884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:35.436899 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:35.437001 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:05:35.437154 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.438385 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.439375 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.440060 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.440138 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:35.440190 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:35.440508 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 15:05:35.440556 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":91,"threads_started":1}
I20260806 15:05:35.440624 22692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:35.436460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:34101 (local address 127.0.0.1:54386)
0806 15:05:35.436590 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:35.436598 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:35.436634 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:35.436996 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:35.436997 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:35.436999 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:05:35.437038 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:35.437040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.438348 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.438349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:35.439407 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:35.439410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.440390 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.440391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:35.440460 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:35.440463 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:05:35.440514 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 15:05:35.440557 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":80,"threads_started":1}
I20260806 15:05:35.440821 22693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:35.436636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54386 (local address 127.0.236.66:34101)
0806 15:05:35.436798 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:35.436802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:35.436813 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:05:35.436851 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:35.436853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:35.436868 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:35.437110 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:05:35.437175 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.438250 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.439493 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.440221 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.440287 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:35.440351 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:35.440709 (+ 358us) server_negotiation.cc:299] Negotiation successful
0806 15:05:35.440751 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
I20260806 15:05:35.441190 22567 tablet_service.cc:1467] Processing CreateTablet for tablet a611aae3f2774ab88698e46ad221e6e5 (DEFAULT_TABLE table=test-table [id=11540116d3414fda89171f2fc00fb954]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:05:35.441362 22380 tablet_service.cc:1467] Processing CreateTablet for tablet a611aae3f2774ab88698e46ad221e6e5 (DEFAULT_TABLE table=test-table [id=11540116d3414fda89171f2fc00fb954]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:05:35.441457 22567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a611aae3f2774ab88698e46ad221e6e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:05:35.441648 22380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a611aae3f2774ab88698e46ad221e6e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:05:35.444000 22696 tablet_bootstrap.cc:492] T a611aae3f2774ab88698e46ad221e6e5 P 925546e3fb0c4e378b64d8715107c0ef: Bootstrap starting.
I20260806 15:05:35.444175 22697 tablet_bootstrap.cc:492] T a611aae3f2774ab88698e46ad221e6e5 P 0be9946ecc994c248f9ac8a34da06763: Bootstrap starting.
I20260806 15:05:35.444796 22696 tablet_bootstrap.cc:654] T a611aae3f2774ab88698e46ad221e6e5 P 925546e3fb0c4e378b64d8715107c0ef: Neither blocks nor log segments found. Creating new log.
I20260806 15:05:35.444796 22697 tablet_bootstrap.cc:654] T a611aae3f2774ab88698e46ad221e6e5 P 0be9946ecc994c248f9ac8a34da06763: Neither blocks nor log segments found. Creating new log.
I20260806 15:05:35.445346 22696 log.cc:824] T a611aae3f2774ab88698e46ad221e6e5 P 925546e3fb0c4e378b64d8715107c0ef: Log is configured to *not* fsync() on all Append() calls
I20260806 15:05:35.445346 22697 log.cc:824] T a611aae3f2774ab88698e46ad221e6e5 P 0be9946ecc994c248f9ac8a34da06763: Log is configured to *not* fsync() on all Append() calls
I20260806 15:05:35.446051 22697 tablet_bootstrap.cc:492] T a611aae3f2774ab88698e46ad221e6e5 P 0be9946ecc994c248f9ac8a34da06763: No bootstrap required, opened a new log
I20260806 15:05:35.446062 22696 tablet_bootstrap.cc:492] T a611aae3f2774ab88698e46ad221e6e5 P 925546e3fb0c4e378b64d8715107c0ef: No bootstrap required, opened a new log
I20260806 15:05:35.446107 22697 ts_tablet_manager.cc:1397] T a611aae3f2774ab88698e46ad221e6e5 P 0be9946ecc994c248f9ac8a34da06763: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:05:35.446113 22696 ts_tablet_manager.cc:1397] T a611aae3f2774ab88698e46ad221e6e5 P 925546e3fb0c4e378b64d8715107c0ef: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:05:35.447444 22697 raft_consensus.cc:348] T a611aae3f2774ab88698e46ad221e6e5 P 0be9946ecc994c248f9ac8a34da06763 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:35.447567 22697 raft_consensus.cc:374] T a611aae3f2774ab88698e46ad221e6e5 P 0be9946ecc994c248f9ac8a34da06763 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:05:35.447588 22697 raft_consensus.cc:729] T a611aae3f2774ab88698e46ad221e6e5 P 0be9946ecc994c248f9ac8a34da06763 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0be9946ecc994c248f9ac8a34da06763, State: Initialized, Role: FOLLOWER
I20260806 15:05:35.447535 22696 raft_consensus.cc:348] T a611aae3f2774ab88698e46ad221e6e5 P 925546e3fb0c4e378b64d8715107c0ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:35.447690 22696 raft_consensus.cc:374] T a611aae3f2774ab88698e46ad221e6e5 P 925546e3fb0c4e378b64d8715107c0ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:05:35.447680 22697 consensus_queue.cc:260] T a611aae3f2774ab88698e46ad221e6e5 P 0be9946ecc994c248f9ac8a34da06763 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:35.447733 22696 raft_consensus.cc:729] T a611aae3f2774ab88698e46ad221e6e5 P 925546e3fb0c4e378b64d8715107c0ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 925546e3fb0c4e378b64d8715107c0ef, State: Initialized, Role: FOLLOWER
I20260806 15:05:35.447829 22696 consensus_queue.cc:260] T a611aae3f2774ab88698e46ad221e6e5 P 925546e3fb0c4e378b64d8715107c0ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:35.447944 22482 heartbeater.cc:503] Master 127.0.236.126:39761 was elected leader, sending a full tablet report...
I20260806 15:05:35.448082 22696 ts_tablet_manager.cc:1428] T a611aae3f2774ab88698e46ad221e6e5 P 925546e3fb0c4e378b64d8715107c0ef: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:05:35.448082 22684 heartbeater.cc:503] Master 127.0.236.126:39761 was elected leader, sending a full tablet report...
I20260806 15:05:35.447944 22697 ts_tablet_manager.cc:1428] T a611aae3f2774ab88698e46ad221e6e5 P 0be9946ecc994c248f9ac8a34da06763: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:05:35.507900 22282 raft_consensus.cc:482] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:05:35.508035 22282 raft_consensus.cc:504] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:35.508355 22282 leader_election.cc:288] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 925546e3fb0c4e378b64d8715107c0ef (127.0.236.67:46099), 0be9946ecc994c248f9ac8a34da06763 (127.0.236.66:34101)
I20260806 15:05:35.512104 22693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:35.508698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:35123 (local address 127.0.236.66:34101)
0806 15:05:35.508736 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:35.508738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:35.508798 (+ 60us) server_negotiation.cc:407] Connection header received
0806 15:05:35.508852 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:35.508853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:35.508861 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:35.508909 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:05:35.509044 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.509741 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.511166 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.511747 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.511817 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:35.511827 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:35.511999 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 15:05:35.512036 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 15:05:35.512104 22691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:35.508564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:37207 (local address 127.0.236.67:46099)
0806 15:05:35.508615 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:35.508618 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:35.508628 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:05:35.508748 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:35.508749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:35.508758 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:35.508821 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:05:35.508976 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.509705 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.511166 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.511747 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.511820 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:35.511827 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:35.511999 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 15:05:35.512037 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 15:05:35.512115 22275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:35.508505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:46099 (local address 127.0.236.65:37207)
0806 15:05:35.508554 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:35.508565 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:35.508616 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:35.508834 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:35.508835 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:35.508837 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:05:35.508908 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:35.508909 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.509794 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.509795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:35.511074 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:35.511078 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.511826 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.511827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:35.511907 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:35.511933 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 15:05:35.511987 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 15:05:35.512028 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":51}
I20260806 15:05:35.512123 22705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:35.508602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:34101 (local address 127.0.236.65:35123)
0806 15:05:35.508756 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:35.508763 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:35.508806 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:35.508927 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:35.508928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:35.508929 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:05:35.508988 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:35.508989 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.509805 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.509806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:35.511074 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:35.511078 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.511843 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.511844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:35.511907 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:35.511934 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 15:05:35.511987 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 15:05:35.512028 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":7,"thread_start_us":110,"threads_started":1}
I20260806 15:05:35.512455 22576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a611aae3f2774ab88698e46ad221e6e5" candidate_uuid: "5e08410ceb93456ea29bb3eb223bd15f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "925546e3fb0c4e378b64d8715107c0ef" is_pre_election: true
I20260806 15:05:35.512590 22576 raft_consensus.cc:2454] T a611aae3f2774ab88698e46ad221e6e5 P 925546e3fb0c4e378b64d8715107c0ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e08410ceb93456ea29bb3eb223bd15f in term 0.
I20260806 15:05:35.512748 22412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a611aae3f2774ab88698e46ad221e6e5" candidate_uuid: "5e08410ceb93456ea29bb3eb223bd15f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0be9946ecc994c248f9ac8a34da06763" is_pre_election: true
I20260806 15:05:35.512856 22412 raft_consensus.cc:2454] T a611aae3f2774ab88698e46ad221e6e5 P 0be9946ecc994c248f9ac8a34da06763 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e08410ceb93456ea29bb3eb223bd15f in term 0.
I20260806 15:05:35.513063 22084 leader_election.cc:302] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e08410ceb93456ea29bb3eb223bd15f, 925546e3fb0c4e378b64d8715107c0ef; no voters:
I20260806 15:05:35.513191 22282 raft_consensus.cc:2781] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:05:35.513257 22282 raft_consensus.cc:482] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:05:35.513280 22282 raft_consensus.cc:3037] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:05:35.513964 22282 raft_consensus.cc:504] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:35.514077 22282 leader_election.cc:288] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 1 election: Requested vote from peers 925546e3fb0c4e378b64d8715107c0ef (127.0.236.67:46099), 0be9946ecc994c248f9ac8a34da06763 (127.0.236.66:34101)
I20260806 15:05:35.514250 22412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a611aae3f2774ab88698e46ad221e6e5" candidate_uuid: "5e08410ceb93456ea29bb3eb223bd15f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0be9946ecc994c248f9ac8a34da06763"
I20260806 15:05:35.514285 22576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a611aae3f2774ab88698e46ad221e6e5" candidate_uuid: "5e08410ceb93456ea29bb3eb223bd15f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "925546e3fb0c4e378b64d8715107c0ef"
I20260806 15:05:35.514318 22412 raft_consensus.cc:3037] T a611aae3f2774ab88698e46ad221e6e5 P 0be9946ecc994c248f9ac8a34da06763 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:05:35.514350 22576 raft_consensus.cc:3037] T a611aae3f2774ab88698e46ad221e6e5 P 925546e3fb0c4e378b64d8715107c0ef [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:05:35.515004 22412 raft_consensus.cc:2454] T a611aae3f2774ab88698e46ad221e6e5 P 0be9946ecc994c248f9ac8a34da06763 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e08410ceb93456ea29bb3eb223bd15f in term 1.
I20260806 15:05:35.515019 22576 raft_consensus.cc:2454] T a611aae3f2774ab88698e46ad221e6e5 P 925546e3fb0c4e378b64d8715107c0ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e08410ceb93456ea29bb3eb223bd15f in term 1.
I20260806 15:05:35.515162 22084 leader_election.cc:302] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e08410ceb93456ea29bb3eb223bd15f, 925546e3fb0c4e378b64d8715107c0ef; no voters:
I20260806 15:05:35.515247 22282 raft_consensus.cc:2781] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:05:35.515388 22282 raft_consensus.cc:686] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 LEADER]: Becoming Leader. State: Replica: 5e08410ceb93456ea29bb3eb223bd15f, State: Running, Role: LEADER
I20260806 15:05:35.515444 22282 consensus_queue.cc:237] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:35.516116 21952 catalog_manager.cc:5495] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e08410ceb93456ea29bb3eb223bd15f (127.0.236.65). New cstate: current_term: 1 leader_uuid: "5e08410ceb93456ea29bb3eb223bd15f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } health_report { overall_health: HEALTHY } } }
W20260806 15:05:35.529220 22483 tablet.cc:2369] T a611aae3f2774ab88698e46ad221e6e5 P 0be9946ecc994c248f9ac8a34da06763: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:05:35.532977 22285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:35.530703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45660 (local address 127.0.236.65:34465)
0806 15:05:35.530738 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:35.530740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:35.530748 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:05:35.530816 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:35.530816 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:35.530834 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:35.530894 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:05:35.531000 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.531516 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.532251 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:35.532470 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:35.532526 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:35.532667 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:05:35.532762 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:05:35.532818 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:35.532869 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 15:05:35.532888 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:05:35.534179 22412 raft_consensus.cc:1261] T a611aae3f2774ab88698e46ad221e6e5 P 0be9946ecc994c248f9ac8a34da06763 [term 1 FOLLOWER]: Refusing update from remote peer 5e08410ceb93456ea29bb3eb223bd15f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:05:35.534214 22576 raft_consensus.cc:1261] T a611aae3f2774ab88698e46ad221e6e5 P 925546e3fb0c4e378b64d8715107c0ef [term 1 FOLLOWER]: Refusing update from remote peer 5e08410ceb93456ea29bb3eb223bd15f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:05:35.534453 22282 consensus_queue.cc:1035] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:05:35.534569 22706 consensus_queue.cc:1035] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [LEADER]: Connected to new peer: Peer: permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:05:35.537357 22711 mvcc.cc:205] Tried to move back new op lower bound from 7315573700746031104 to 7315573700671680512. Current Snapshot: MvccSnapshot[applied={T|T < 7315573700746031104 or (T in {7315573700746031104})}]
I20260806 15:05:35.538606 21952 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:44576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:05:35.538684 21952 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:44576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:05:35.539268 21952 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f: Sending DeleteTablet for 3 replicas of tablet a611aae3f2774ab88698e46ad221e6e5
I20260806 15:05:35.539521 22567 tablet_service.cc:1514] Processing DeleteTablet for tablet a611aae3f2774ab88698e46ad221e6e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:05:35 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:51628
I20260806 15:05:35.539563 22129 tablet_service.cc:1514] Processing DeleteTablet for tablet a611aae3f2774ab88698e46ad221e6e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:05:35 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:45654
I20260806 15:05:35.539572 22380 tablet_service.cc:1514] Processing DeleteTablet for tablet a611aae3f2774ab88698e46ad221e6e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:05:35 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:54386
I20260806 15:05:35.539646 22287 tablet_replica.cc:330] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f: stopping tablet replica
I20260806 15:05:35.539686 22287 raft_consensus.cc:2229] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:05:35.539757 22287 raft_consensus.cc:2258] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:05:35.539779 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 22029
I20260806 15:05:35.543972 22727 tablet_replica.cc:330] T a611aae3f2774ab88698e46ad221e6e5 P 0be9946ecc994c248f9ac8a34da06763: stopping tablet replica
I20260806 15:05:35.544073 22727 raft_consensus.cc:2229] T a611aae3f2774ab88698e46ad221e6e5 P 0be9946ecc994c248f9ac8a34da06763 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:05:35.544162 22727 raft_consensus.cc:2258] T a611aae3f2774ab88698e46ad221e6e5 P 0be9946ecc994c248f9ac8a34da06763 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:05:35.545177 22727 ts_tablet_manager.cc:1905] T a611aae3f2774ab88698e46ad221e6e5 P 0be9946ecc994c248f9ac8a34da06763: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:05:35.546042 22726 tablet_replica.cc:330] T a611aae3f2774ab88698e46ad221e6e5 P 925546e3fb0c4e378b64d8715107c0ef: stopping tablet replica
I20260806 15:05:35.546136 22726 raft_consensus.cc:2229] T a611aae3f2774ab88698e46ad221e6e5 P 925546e3fb0c4e378b64d8715107c0ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:05:35.546218 22726 raft_consensus.cc:2258] T a611aae3f2774ab88698e46ad221e6e5 P 925546e3fb0c4e378b64d8715107c0ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:05:35.546674 22727 ts_tablet_manager.cc:1918] T a611aae3f2774ab88698e46ad221e6e5 P 0be9946ecc994c248f9ac8a34da06763: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:05:35.546731 22727 log.cc:1196] T a611aae3f2774ab88698e46ad221e6e5 P 0be9946ecc994c248f9ac8a34da06763: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/wal/wals/a611aae3f2774ab88698e46ad221e6e5
I20260806 15:05:35.546940 22727 ts_tablet_manager.cc:1939] T a611aae3f2774ab88698e46ad221e6e5 P 0be9946ecc994c248f9ac8a34da06763: Deleting consensus metadata
I20260806 15:05:35.547397 21934 catalog_manager.cc:4841] TS 0be9946ecc994c248f9ac8a34da06763 (127.0.236.66:34101): tablet a611aae3f2774ab88698e46ad221e6e5 (table test-table [id=11540116d3414fda89171f2fc00fb954]) successfully deleted
I20260806 15:05:35.547564 22726 ts_tablet_manager.cc:1905] T a611aae3f2774ab88698e46ad221e6e5 P 925546e3fb0c4e378b64d8715107c0ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:05:35.548578 22726 ts_tablet_manager.cc:1918] T a611aae3f2774ab88698e46ad221e6e5 P 925546e3fb0c4e378b64d8715107c0ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:05:35.548640 22726 log.cc:1196] T a611aae3f2774ab88698e46ad221e6e5 P 925546e3fb0c4e378b64d8715107c0ef: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/a611aae3f2774ab88698e46ad221e6e5
I20260806 15:05:35.548846 22726 ts_tablet_manager.cc:1939] T a611aae3f2774ab88698e46ad221e6e5 P 925546e3fb0c4e378b64d8715107c0ef: Deleting consensus metadata
I20260806 15:05:35.549185 21934 catalog_manager.cc:4841] TS 925546e3fb0c4e378b64d8715107c0ef (127.0.236.67:46099): tablet a611aae3f2774ab88698e46ad221e6e5 (table test-table [id=11540116d3414fda89171f2fc00fb954]) successfully deleted
W20260806 15:05:35.552560 21936 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.236.65:34465 (error 108)
I20260806 15:05:35.552978 22692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:35.552860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:34465 (local address 127.0.0.1:45670)
0806 15:05:35.552907 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:35.552945 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.65:34465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 15:05:35.553000 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 22280
W20260806 15:05:35.553107 21936 catalog_manager.cc:4584] TS 5e08410ceb93456ea29bb3eb223bd15f (127.0.236.65:34465): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a611aae3f2774ab88698e46ad221e6e5: Network error: Client connection negotiation failed: client connection to 127.0.236.65:34465: connect: Connection refused (error 111)
I20260806 15:05:35.561190 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 22485
I20260806 15:05:35.573618 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 21912
I20260806 15:05:35.580216 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:39761
--webserver_interface=127.0.236.126
--webserver_port=40577
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:35035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:39761
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:05:35.658666 22729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:35.658849 22729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:35.658869 22729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:35.660025 22729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:05:35.660063 22729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:35.660075 22729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:35.660087 22729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:05:35.660110 22729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:05:35.661579 22729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:35035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:39761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:39761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=40577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:35.661798 22729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:35.662024 22729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 15:05:35.666397 22729 server_base.cc:1034] running on GCE node
W20260806 15:05:35.666332 22744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:35.666357 22742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:35.666344 22739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:35.666762 22729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:35.667013 22729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:35.668146 22729 hybrid_clock.cc:648] HybridClock initialized: now 1786028735668132 us; error 26 us; skew 500 ppm
Aug 06 15:05:35 dist-test-slave-tc2s krb5kdc[21069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028735, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:35.670418 22729 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:05:35.670770 22729 webserver.cc:460] Webserver started at http://127.0.236.126:40577/ using document root <none> and password file <none>
I20260806 15:05:35.670928 22729 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:35.670964 22729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:05:35.672112 22729 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:05:35.672679 22753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:35.672842 22729 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:05:35.672923 22729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "e58f8c49b33846ea9e314f0723da7a9f"
format_stamp: "Formatted at 2026-08-06 15:05:33 on dist-test-slave-tc2s"
server_key: "5d9a00fe1a4ab734dae8286e929fbb7f"
server_key_iv: "43e6c129b190aab570e4fb18c8076d55"
server_key_version: "encryptionkey@0"
I20260806 15:05:35.673210 22729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:35.718530 22729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:35.720127 22729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:35.720228 22729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:35.723919 22729 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:39761
I20260806 15:05:35.723961 22829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:39761 every 8 connection(s)
I20260806 15:05:35.724290 22729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:05:35.724771 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 22729
I20260806 15:05:35.725023 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:34465
--local_ip_for_outbound_sockets=127.0.236.65
--tserver_master_addrs=127.0.236.126:39761
--webserver_port=37915
--webserver_interface=127.0.236.65
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:35035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:05:35.725838 22830 sys_catalog.cc:263] Verifying existing consensus state
I20260806 15:05:35.726550 22830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f: Bootstrap starting.
I20260806 15:05:35.734862 22830 log.cc:824] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f: Log is configured to *not* fsync() on all Append() calls
I20260806 15:05:35.736972 22830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 15:05:35.737214 22830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f: Bootstrap complete.
I20260806 15:05:35.739241 22830 raft_consensus.cc:348] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e58f8c49b33846ea9e314f0723da7a9f" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39761 } }
I20260806 15:05:35.739457 22830 raft_consensus.cc:729] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e58f8c49b33846ea9e314f0723da7a9f, State: Initialized, Role: FOLLOWER
I20260806 15:05:35.739594 22830 consensus_queue.cc:260] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e58f8c49b33846ea9e314f0723da7a9f" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39761 } }
I20260806 15:05:35.739650 22830 raft_consensus.cc:388] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:05:35.739686 22830 raft_consensus.cc:482] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:05:35.739732 22830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [term 2 FOLLOWER]: Advancing to term 3
I20260806 15:05:35.740655 22830 raft_consensus.cc:504] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e58f8c49b33846ea9e314f0723da7a9f" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39761 } }
I20260806 15:05:35.740741 22830 leader_election.cc:302] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e58f8c49b33846ea9e314f0723da7a9f; no voters:
I20260806 15:05:35.740887 22830 leader_election.cc:288] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 15:05:35.740934 22837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [term 3 FOLLOWER]: Leader election won for term 3
I20260806 15:05:35.741058 22837 raft_consensus.cc:686] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [term 3 LEADER]: Becoming Leader. State: Replica: e58f8c49b33846ea9e314f0723da7a9f, State: Running, Role: LEADER
I20260806 15:05:35.741151 22830 sys_catalog.cc:564] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:05:35.741155 22837 consensus_queue.cc:237] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e58f8c49b33846ea9e314f0723da7a9f" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39761 } }
I20260806 15:05:35.741438 22838 sys_catalog.cc:455] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e58f8c49b33846ea9e314f0723da7a9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e58f8c49b33846ea9e314f0723da7a9f" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39761 } } }
I20260806 15:05:35.741535 22838 sys_catalog.cc:458] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [sys.catalog]: This master's current role is: LEADER
I20260806 15:05:35.741451 22839 sys_catalog.cc:455] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e58f8c49b33846ea9e314f0723da7a9f. Latest consensus state: current_term: 3 leader_uuid: "e58f8c49b33846ea9e314f0723da7a9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e58f8c49b33846ea9e314f0723da7a9f" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39761 } } }
I20260806 15:05:35.741852 22839 sys_catalog.cc:458] T 00000000000000000000000000000000 P e58f8c49b33846ea9e314f0723da7a9f [sys.catalog]: This master's current role is: LEADER
I20260806 15:05:35.741935 22845 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:05:35.742736 22845 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:05:35.742846 22845 catalog_manager.cc:1259] Loaded cluster ID: 43da28a2a2de450ead475558254a73e2
I20260806 15:05:35.742955 22845 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 15:05:35.743324 22845 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f2c228709ad google::LogMessage::Fail() at ??:0
@ 0x7f2c2287388a google::LogMessage::SendToLog() at ??:0
@ 0x7f2c228704de google::LogMessage::Flush() at ??:0
@ 0x7f2c22871a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f2c2a7910f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f2c2a79a8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f2c2473e0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f2c24733954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2c255976db start_thread at ??:0
@ 0x7f2c2136e61f clone at ??:0
W20260806 15:05:35.836622 22834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:35.836877 22834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:35.836913 22834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:35.838604 22834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:35.838662 22834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:35.838708 22834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:05:35.840783 22834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:35035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:34465
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=37915
--tserver_master_addrs=127.0.236.126:39761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:35.841017 22834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:35.841276 22834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:05:35.846287 22888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:35.846282 22889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:35.846280 22892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:35.846536 22834 server_base.cc:1034] running on GCE node
I20260806 15:05:35.846717 22834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:35.846937 22834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:35.848076 22834 hybrid_clock.cc:648] HybridClock initialized: now 1786028735848063 us; error 30 us; skew 500 ppm
Aug 06 15:05:35 dist-test-slave-tc2s krb5kdc[21069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028735, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:35.850518 22834 init.cc:377] Logged in from keytab as kudu/127.0.236.65@KRBTEST.COM (short username kudu)
I20260806 15:05:35.850903 22834 webserver.cc:460] Webserver started at http://127.0.236.65:37915/ using document root <none> and password file <none>
I20260806 15:05:35.851114 22834 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:35.851155 22834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:05:35.852396 22834 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:05:35.853055 22899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:35.853209 22834 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:35.853281 22834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "5e08410ceb93456ea29bb3eb223bd15f"
format_stamp: "Formatted at 2026-08-06 15:05:34 on dist-test-slave-tc2s"
server_key: "77e5c0221ea6f3457fe584f6278b8acf"
server_key_iv: "958e45693423899335221e41aeaa23db"
server_key_version: "encryptionkey@0"
I20260806 15:05:35.853677 22834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:35.867622 22834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:35.869740 22834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:35.869884 22834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:35.870143 22834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:05:35.870534 22911 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 15:05:35.872277 22834 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 15:05:35.872318 22834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 15:05:35.872346 22834 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 15:05:35.872915 22834 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 15:05:35.872948 22834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 15:05:35.873029 22911 tablet_bootstrap.cc:492] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f: Bootstrap starting.
I20260806 15:05:35.879380 22834 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:34465
I20260806 15:05:35.879474 23073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:34465 every 8 connection(s)
I20260806 15:05:35.879745 22834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:05:35.880604 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 22834
I20260806 15:05:35.880857 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:34101
--local_ip_for_outbound_sockets=127.0.236.66
--tserver_master_addrs=127.0.236.126:39761
--webserver_port=33159
--webserver_interface=127.0.236.66
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:35035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:05:35.883560 22911 log.cc:824] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f: Log is configured to *not* fsync() on all Append() calls
I20260806 15:05:35.885430 22911 tablet_bootstrap.cc:492] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 15:05:35.885713 22911 tablet_bootstrap.cc:492] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f: Bootstrap complete.
I20260806 15:05:35.886507 22911 ts_tablet_manager.cc:1397] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.010s
I20260806 15:05:35.887455 22911 raft_consensus.cc:348] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:35.887670 22911 raft_consensus.cc:729] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e08410ceb93456ea29bb3eb223bd15f, State: Initialized, Role: FOLLOWER
I20260806 15:05:35.887806 22911 consensus_queue.cc:260] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:35.888046 22911 ts_tablet_manager.cc:1428] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
W20260806 15:05:35.955519 23079 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:05:35.880346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.65:47109)
0806 15:05:35.882466 (+ 2120us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:35.882516 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:35.883048 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:35.955316 (+ 72268us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":2065,"thread_start_us":74,"threads_started":1}
W20260806 15:05:35.955753 23077 heartbeater.cc:644] Failed to heartbeat to 127.0.236.126:39761 (0 consecutive failures): Network error: Failed to ping master at 127.0.236.126:39761: Client connection negotiation failed: client connection to 127.0.236.126:39761: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 15:05:35.956087 23079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:35.955980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.65:51095)
0806 15:05:35.956029 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:35.956069 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 15:05:35.956427 23079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:35.956321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.65:41267)
0806 15:05:35.956388 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:35.956410 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 15:05:35.956540 23077 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 15:05:35.975459 23080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:35.975643 23080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:35.975677 23080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:35.976805 23080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:35.976845 23080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:35.976881 23080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:05:35.978355 23080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:35035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:34101
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=33159
--tserver_master_addrs=127.0.236.126:39761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:35.978581 23080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:35.978770 23080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:05:35.983363 23093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:35.983366 23097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:35.983412 23080 server_base.cc:1034] running on GCE node
W20260806 15:05:35.983371 23094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:35.983681 23080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:35.983917 23080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:35.985076 23080 hybrid_clock.cc:648] HybridClock initialized: now 1786028735985062 us; error 29 us; skew 500 ppm
Aug 06 15:05:35 dist-test-slave-tc2s krb5kdc[21069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028735, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:35.987471 23080 init.cc:377] Logged in from keytab as kudu/127.0.236.66@KRBTEST.COM (short username kudu)
I20260806 15:05:35.987866 23080 webserver.cc:460] Webserver started at http://127.0.236.66:33159/ using document root <none> and password file <none>
I20260806 15:05:35.988047 23080 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:35.988086 23080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:05:35.989207 23080 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:05:35.989703 23106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:35.989852 23080 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:35.989966 23080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "0be9946ecc994c248f9ac8a34da06763"
format_stamp: "Formatted at 2026-08-06 15:05:34 on dist-test-slave-tc2s"
server_key: "bc6eee00b36ea72367bd00dfdd03aeff"
server_key_iv: "7a5c48eb9584ca18f4053cd9878ab378"
server_key_version: "encryptionkey@0"
I20260806 15:05:35.990249 23080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:35.997303 23080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:35.998871 23080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:35.998996 23080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:35.999176 23080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:05:35.999428 23080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:05:35.999461 23080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:35.999481 23080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:05:35.999504 23080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:36.005005 23080 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:34101
I20260806 15:05:36.005057 23272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:34101 every 8 connection(s)
I20260806 15:05:36.005335 23080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
I20260806 15:05:36.006276 23275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:36.005937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.66:42355)
0806 15:05:36.006088 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:36.006151 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
W20260806 15:05:36.006634 23273 heartbeater.cc:644] Failed to heartbeat to 127.0.236.126:39761 (0 consecutive failures): Network error: Failed to ping master at 127.0.236.126:39761: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
I20260806 15:05:36.006856 23275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:36.006780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.66:60795)
0806 15:05:36.006816 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:36.006841 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260806 15:05:36.007063 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 23080
I20260806 15:05:36.007279 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:46099
--local_ip_for_outbound_sockets=127.0.236.67
--tserver_master_addrs=127.0.236.126:39761
--webserver_port=37337
--webserver_interface=127.0.236.67
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:35035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:05:36.008832 23275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:36.008751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.66:55251)
0806 15:05:36.008790 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:36.008813 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 15:05:36.008937 23273 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 15:05:36.086477 23276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:36.086668 23276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:36.086688 23276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:36.087738 23276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:36.087774 23276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:36.087797 23276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:05:36.089159 23276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:35035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:46099
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=37337
--tserver_master_addrs=127.0.236.126:39761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:36.089326 23276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:36.089506 23276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:05:36.094532 23283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:36.094820 23284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:36.094658 23286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:36.094884 23276 server_base.cc:1034] running on GCE node
I20260806 15:05:36.095116 23276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:36.095340 23276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:36.096473 23276 hybrid_clock.cc:648] HybridClock initialized: now 1786028736096464 us; error 25 us; skew 500 ppm
Aug 06 15:05:36 dist-test-slave-tc2s krb5kdc[21069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028736, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:36.098701 23276 init.cc:377] Logged in from keytab as kudu/127.0.236.67@KRBTEST.COM (short username kudu)
I20260806 15:05:36.099102 23276 webserver.cc:460] Webserver started at http://127.0.236.67:37337/ using document root <none> and password file <none>
I20260806 15:05:36.099285 23276 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:36.099334 23276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:05:36.100522 23276 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:05:36.101001 23295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:36.101133 23276 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:36.101195 23276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "925546e3fb0c4e378b64d8715107c0ef"
format_stamp: "Formatted at 2026-08-06 15:05:34 on dist-test-slave-tc2s"
server_key: "98516631d3b8cc386d30c57e1567eee6"
server_key_iv: "e2b373d9074aecd6cbccff6bf3f5c53e"
server_key_version: "encryptionkey@0"
I20260806 15:05:36.101495 23276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:36.132934 23276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:36.134931 23276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:36.135054 23276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:36.135250 23276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:05:36.135529 23276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:05:36.135563 23276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:36.135591 23276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:05:36.135607 23276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:36.140818 23276 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:46099
I20260806 15:05:36.140880 23469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:46099 every 8 connection(s)
I20260806 15:05:36.141157 23276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
I20260806 15:05:36.142093 23472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:36.141732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.67:55057)
0806 15:05:36.141862 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:36.141950 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":65,"threads_started":1}
I20260806 15:05:36.142163 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 23276
W20260806 15:05:36.142445 23470 heartbeater.cc:644] Failed to heartbeat to 127.0.236.126:39761 (0 consecutive failures): Network error: Failed to ping master at 127.0.236.126:39761: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
I20260806 15:05:36.142722 23472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:36.142630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.67:52815)
0806 15:05:36.142671 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:36.142699 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 15:05:36.143018 23472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:36.142934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.67:39253)
0806 15:05:36.142972 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:36.142996 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 15:05:36.143108 23470 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 15:05:36.957051 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:36.956810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.65:51263)
0806 15:05:36.956961 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:36.956990 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":69,"threads_started":1}
I20260806 15:05:37.009371 23475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:37.009187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.66:40139)
0806 15:05:37.009296 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:37.009327 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":56,"threads_started":1}
I20260806 15:05:37.143545 23477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:37.143373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.67:50381)
0806 15:05:37.143469 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:37.143499 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":53,"threads_started":1}
I20260806 15:05:37.566707 23480 raft_consensus.cc:482] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:05:37.566826 23480 raft_consensus.cc:504] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:37.567149 23480 leader_election.cc:288] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 925546e3fb0c4e378b64d8715107c0ef (127.0.236.67:46099), 0be9946ecc994c248f9ac8a34da06763 (127.0.236.66:34101)
Aug 06 15:05:37 dist-test-slave-tc2s krb5kdc[21069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028735, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for kudu/127.0.236.66@KRBTEST.COM
Aug 06 15:05:37 dist-test-slave-tc2s krb5kdc[21069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028735, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for kudu/127.0.236.67@KRBTEST.COM
I20260806 15:05:37.576915 23484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:37.567384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:47697 (local address 127.0.236.67:46099)
0806 15:05:37.567627 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:37.567638 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:37.567671 (+ 33us) server_negotiation.cc:407] Connection header received
0806 15:05:37.568121 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:37.568123 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:37.568159 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:37.568282 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:37.569030 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:37.569887 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:37.570863 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:37.571259 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:37.571357 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:37.573872 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:37.573876 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:37.573876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:37.573911 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:37.575504 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:37.575841 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:37.575843 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:37.575844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:37.575948 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:37.576123 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:37.576123 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:37.576124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:37.576341 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:37.576408 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:37.576685 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 15:05:37.576805 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":74,"threads_started":1}
I20260806 15:05:37.576903 23482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:37.567410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:34101 (local address 127.0.236.65:35025)
0806 15:05:37.567489 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:37.567497 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:37.567532 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:37.568337 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:37.568340 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:37.568738 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:37.568916 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:37.568922 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:37.569995 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:37.569996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:37.570785 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:37.570787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:37.571344 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:37.571345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:37.571453 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:37.571959 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:37.571967 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:37.573446 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:37.575358 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:37.575366 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:37.575368 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:37.575666 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:37.575931 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:37.575931 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:37.575932 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:37.576053 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:37.576408 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:37.576415 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:37.576533 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 15:05:37.576621 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 15:05:37.576799 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":27,"thread_start_us":39,"threads_started":1}
I20260806 15:05:37.576920 23485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:37.567546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:35025 (local address 127.0.236.66:34101)
0806 15:05:37.567652 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:37.567660 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:37.567677 (+ 17us) server_negotiation.cc:407] Connection header received
0806 15:05:37.568098 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:37.568100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:37.568140 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:37.568270 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:37.569045 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:37.569879 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:37.570870 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:37.571260 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:37.571362 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:37.573543 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:37.573547 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:37.573547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:37.573556 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:37.575277 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:37.575744 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:37.575747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:37.575747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:37.575857 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:37.576111 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:37.576112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:37.576112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:37.576341 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:37.576413 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:37.576687 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 15:05:37.576810 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":38,"threads_started":1}
I20260806 15:05:37.577566 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:37.567260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:46099 (local address 127.0.236.65:47697)
0806 15:05:37.567382 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:37.567393 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:37.567461 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:37.568337 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:37.568341 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:37.568737 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:37.568918 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:37.568923 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:37.570023 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:37.570024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:37.570784 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:37.570786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:37.571341 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:37.571342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:37.571453 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:37.572012 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:37.572017 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:37.573780 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:37.575577 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:37.575578 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:37.575578 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:37.575764 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:37.576011 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:37.576012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:37.576012 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:37.576066 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:37.576404 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:37.576412 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:37.576533 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 15:05:37.576621 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 15:05:37.576872 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":152,"thread_start_us":49,"threads_started":1}
W20260806 15:05:37.577620 22902 leader_election.cc:341] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0be9946ecc994c248f9ac8a34da06763 (127.0.236.66:34101): Not found: Tablet not found: a611aae3f2774ab88698e46ad221e6e5
W20260806 15:05:37.578230 22902 leader_election.cc:341] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 925546e3fb0c4e378b64d8715107c0ef (127.0.236.67:46099): Not found: Tablet not found: a611aae3f2774ab88698e46ad221e6e5
I20260806 15:05:37.578285 22902 leader_election.cc:302] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e08410ceb93456ea29bb3eb223bd15f; no voters: 0be9946ecc994c248f9ac8a34da06763, 925546e3fb0c4e378b64d8715107c0ef
I20260806 15:05:37.578380 23480 raft_consensus.cc:2726] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 15:05:37.957552 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:37.957435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.65:35435)
0806 15:05:37.957480 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:37.957512 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 15:05:38.009976 23486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:38.009749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.66:54119)
0806 15:05:38.009859 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:38.009926 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":59,"threads_started":1}
I20260806 15:05:38.144021 23489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:38.143854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.67:57415)
0806 15:05:38.143956 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:38.143984 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":53,"threads_started":1}
I20260806 15:05:38.958165 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:38.957976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.65:45221)
0806 15:05:38.958083 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:38.958119 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":60,"threads_started":1}
I20260806 15:05:39.010533 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:39.010346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.66:35627)
0806 15:05:39.010460 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:39.010495 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":74,"threads_started":1}
I20260806 15:05:39.094765 23492 raft_consensus.cc:482] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:05:39.094844 23492 raft_consensus.cc:504] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:39.094970 23492 leader_election.cc:288] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 925546e3fb0c4e378b64d8715107c0ef (127.0.236.67:46099), 0be9946ecc994c248f9ac8a34da06763 (127.0.236.66:34101)
W20260806 15:05:39.095348 22902 leader_election.cc:341] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0be9946ecc994c248f9ac8a34da06763 (127.0.236.66:34101): Not found: Tablet not found: a611aae3f2774ab88698e46ad221e6e5
W20260806 15:05:39.095419 22902 leader_election.cc:341] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 925546e3fb0c4e378b64d8715107c0ef (127.0.236.67:46099): Not found: Tablet not found: a611aae3f2774ab88698e46ad221e6e5
I20260806 15:05:39.095444 22902 leader_election.cc:302] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e08410ceb93456ea29bb3eb223bd15f; no voters: 0be9946ecc994c248f9ac8a34da06763, 925546e3fb0c4e378b64d8715107c0ef
I20260806 15:05:39.095525 23492 raft_consensus.cc:2726] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 15:05:39.144520 23496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:39.144347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.67:47687)
0806 15:05:39.144445 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:39.144475 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":55,"threads_started":1}
I20260806 15:05:39.958699 23498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:39.958510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.65:42249)
0806 15:05:39.958620 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:39.958654 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":62,"threads_started":1}
I20260806 15:05:40.011107 23499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:40.010906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.66:59953)
0806 15:05:40.011023 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:40.011054 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
I20260806 15:05:40.145021 23503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:40.144852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.67:47005)
0806 15:05:40.144955 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:40.144986 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":56,"threads_started":1}
I20260806 15:05:40.692485 23504 raft_consensus.cc:482] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:05:40.692610 23504 raft_consensus.cc:504] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:40.692767 23504 leader_election.cc:288] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 925546e3fb0c4e378b64d8715107c0ef (127.0.236.67:46099), 0be9946ecc994c248f9ac8a34da06763 (127.0.236.66:34101)
W20260806 15:05:40.693125 22902 leader_election.cc:341] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0be9946ecc994c248f9ac8a34da06763 (127.0.236.66:34101): Not found: Tablet not found: a611aae3f2774ab88698e46ad221e6e5
W20260806 15:05:40.693198 22902 leader_election.cc:341] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 925546e3fb0c4e378b64d8715107c0ef (127.0.236.67:46099): Not found: Tablet not found: a611aae3f2774ab88698e46ad221e6e5
I20260806 15:05:40.693214 22902 leader_election.cc:302] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e08410ceb93456ea29bb3eb223bd15f; no voters: 0be9946ecc994c248f9ac8a34da06763, 925546e3fb0c4e378b64d8715107c0ef
I20260806 15:05:40.693300 23504 raft_consensus.cc:2726] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 15:05:40.959375 23506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:40.959139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.65:57841)
0806 15:05:40.959275 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:40.959321 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":59,"threads_started":1}
I20260806 15:05:41.011685 23507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:41.011467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.66:48775)
0806 15:05:41.011590 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:41.011628 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
I20260806 15:05:41.145524 23508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:41.145337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.67:49075)
0806 15:05:41.145452 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:41.145485 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":66,"threads_started":1}
I20260806 15:05:41.959988 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:41.959792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.65:60443)
0806 15:05:41.959906 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:41.959942 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":62,"threads_started":1}
I20260806 15:05:42.012225 23511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:42.012050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.66:39707)
0806 15:05:42.012158 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:42.012189 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":58,"threads_started":1}
I20260806 15:05:42.146116 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:42.145929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.67:39427)
0806 15:05:42.146039 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:42.146075 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
I20260806 15:05:42.960600 23513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:42.960372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.65:38257)
0806 15:05:42.960498 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:42.960554 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
I20260806 15:05:43.012804 23514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:43.012602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.66:34197)
0806 15:05:43.012723 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:43.012757 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":58,"threads_started":1}
I20260806 15:05:43.146634 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:43.146446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.67:58873)
0806 15:05:43.146565 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:43.146596 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":67,"threads_started":1}
I20260806 15:05:43.961191 23517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:43.960965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.65:35715)
0806 15:05:43.961109 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:43.961146 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
I20260806 15:05:44.013370 23519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:44.013192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.66:48695)
0806 15:05:44.013300 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:44.013330 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":58,"threads_started":1}
I20260806 15:05:44.147195 23520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:44.147000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.67:50367)
0806 15:05:44.147123 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:44.147155 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
I20260806 15:05:44.417511 23523 raft_consensus.cc:482] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:05:44.417606 23523 raft_consensus.cc:504] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:44.417742 23523 leader_election.cc:288] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 925546e3fb0c4e378b64d8715107c0ef (127.0.236.67:46099), 0be9946ecc994c248f9ac8a34da06763 (127.0.236.66:34101)
W20260806 15:05:44.418115 22902 leader_election.cc:341] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 925546e3fb0c4e378b64d8715107c0ef (127.0.236.67:46099): Not found: Tablet not found: a611aae3f2774ab88698e46ad221e6e5
W20260806 15:05:44.418192 22902 leader_election.cc:341] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0be9946ecc994c248f9ac8a34da06763 (127.0.236.66:34101): Not found: Tablet not found: a611aae3f2774ab88698e46ad221e6e5
I20260806 15:05:44.418224 22902 leader_election.cc:302] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e08410ceb93456ea29bb3eb223bd15f; no voters: 0be9946ecc994c248f9ac8a34da06763, 925546e3fb0c4e378b64d8715107c0ef
I20260806 15:05:44.418306 23523 raft_consensus.cc:2726] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 15:05:44.961825 23524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:44.961600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.65:37491)
0806 15:05:44.961726 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:44.961769 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":74,"threads_started":1}
I20260806 15:05:45.013981 23525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:45.013752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.66:42719)
0806 15:05:45.013879 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:45.013940 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":59,"threads_started":1}
I20260806 15:05:45.147722 23526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:45.147538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.67:48613)
0806 15:05:45.147649 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:45.147679 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":67,"threads_started":1}
I20260806 15:05:45.962435 23527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:45.962224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.65:34585)
0806 15:05:45.962350 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:45.962391 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":60,"threads_started":1}
I20260806 15:05:46.014498 23528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:46.014317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.66:47501)
0806 15:05:46.014428 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:46.014459 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":62,"threads_started":1}
I20260806 15:05:46.148227 23529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:46.148042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.67:57553)
0806 15:05:46.148161 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:46.148188 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":68,"threads_started":1}
I20260806 15:05:46.492939 23530 raft_consensus.cc:482] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:05:46.493037 23530 raft_consensus.cc:504] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925546e3fb0c4e378b64d8715107c0ef" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46099 } } peers { permanent_uuid: "0be9946ecc994c248f9ac8a34da06763" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 34101 } } peers { permanent_uuid: "5e08410ceb93456ea29bb3eb223bd15f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 34465 } }
I20260806 15:05:46.493193 23530 leader_election.cc:288] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 925546e3fb0c4e378b64d8715107c0ef (127.0.236.67:46099), 0be9946ecc994c248f9ac8a34da06763 (127.0.236.66:34101)
W20260806 15:05:46.493556 22902 leader_election.cc:341] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0be9946ecc994c248f9ac8a34da06763 (127.0.236.66:34101): Not found: Tablet not found: a611aae3f2774ab88698e46ad221e6e5
W20260806 15:05:46.493650 22902 leader_election.cc:341] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 925546e3fb0c4e378b64d8715107c0ef (127.0.236.67:46099): Not found: Tablet not found: a611aae3f2774ab88698e46ad221e6e5
I20260806 15:05:46.493682 22902 leader_election.cc:302] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e08410ceb93456ea29bb3eb223bd15f; no voters: 0be9946ecc994c248f9ac8a34da06763, 925546e3fb0c4e378b64d8715107c0ef
I20260806 15:05:46.493768 23530 raft_consensus.cc:2726] T a611aae3f2774ab88698e46ad221e6e5 P 5e08410ceb93456ea29bb3eb223bd15f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 15:05:46.962999 23532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:46.962800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.65:60745)
0806 15:05:46.962923 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:46.962954 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":65,"threads_started":1}
I20260806 15:05:47.015000 23535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:47.014835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.66:58027)
0806 15:05:47.014932 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:47.014963 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":53,"threads_started":1}
I20260806 15:05:47.148757 23536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:47.148585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.67:50041)
0806 15:05:47.148687 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:47.148719 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":58,"threads_started":1}
I20260806 15:05:47.963552 23537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:47.963355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.65:42467)
0806 15:05:47.963469 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:47.963506 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":62,"threads_started":1}
I20260806 15:05:48.015534 23538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:48.015344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.66:47903)
0806 15:05:48.015459 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:48.015495 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":55,"threads_started":1}
I20260806 15:05:48.149237 23539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:48.149062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.67:39023)
0806 15:05:48.149168 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:48.149200 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":54,"threads_started":1}
I20260806 15:05:48.964118 23541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:48.963898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.65:35671)
0806 15:05:48.964035 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:48.964072 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260806 15:05:49.016085 23543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:49.015887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.66:39435)
0806 15:05:49.016015 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:49.016047 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
I20260806 15:05:49.149801 23547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:49.149594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.67:56899)
0806 15:05:49.149724 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:49.149759 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":57,"threads_started":1}
I20260806 15:05:49.964665 23548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:49.964465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.65:38849)
0806 15:05:49.964582 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:49.964619 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":63,"threads_started":1}
I20260806 15:05:50.016613 23549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:50.016416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.66:60457)
0806 15:05:50.016543 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:50.016571 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":74,"threads_started":1}
I20260806 15:05:50.150455 23551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:50.150272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.67:49843)
0806 15:05:50.150385 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:50.150415 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":58,"threads_started":1}
I20260806 15:05:50.965296 23554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:50.965103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.65:59071)
0806 15:05:50.965218 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:50.965251 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260806 15:05:51.017179 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:51.017006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.66:56129)
0806 15:05:51.017118 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:51.017144 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":70,"threads_started":1}
I20260806 15:05:51.149447 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 22834
I20260806 15:05:51.153254 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:51.152051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39761 (local address 127.0.236.67:40809)
0806 15:05:51.153173 (+ 1122us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:51.153209 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1106,"thread_start_us":1057,"threads_started":1}
I20260806 15:05:51.162588 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 23080
I20260806 15:05:51.175232 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 23276
2026-08-06T15:05:51Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18506 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:05:51 dist-test-slave-tc2s krb5kdc[23562](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:05:51 dist-test-slave-tc2s krb5kdc[23562](info): set up 2 sockets
Aug 06 15:05:51 dist-test-slave-tc2s krb5kdc[23562](info): commencing operation
krb5kdc: starting...
W20260806 15:05:52.287025 945 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:05:52 dist-test-slave-tc2s krb5kdc[23562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:05:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:05:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:05:52.397311 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:39435
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:39435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:05:52.476707 23577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:52.476917 23577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:52.476948 23577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:52.478098 23577 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 15:05:52.478133 23577 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 15:05:52.478150 23577 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 15:05:52.478165 23577 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:05:52.478180 23577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:52.478199 23577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:52.478219 23577 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:05:52.478238 23577 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:05:52.479707 23577 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:39435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:39435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:52.479945 23577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:52.480145 23577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 15:05:52.484867 23577 server_base.cc:1034] running on GCE node
W20260806 15:05:52.484820 23582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:52.484833 23583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:52.484838 23587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:52.485248 23577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:52.485520 23577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:52.486644 23577 hybrid_clock.cc:648] HybridClock initialized: now 1786028752486630 us; error 31 us; skew 500 ppm
Aug 06 15:05:52 dist-test-slave-tc2s krb5kdc[23562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028752, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:52.492683 23577 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:05:52.493070 23577 webserver.cc:460] Webserver started at http://127.0.236.126:42213/ using document root <none> and password file <none>
I20260806 15:05:52.493255 23577 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:52.493294 23577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:05:52.493384 23577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:05:52.494397 23577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "b87064a86f4b49fe85d745118a59df24"
format_stamp: "Formatted at 2026-08-06 15:05:52 on dist-test-slave-tc2s"
server_key: "b1875e549dcda48f5344a81eed1dbc1e"
server_key_iv: "38cad7a2209da84bd4af2e6f2a352fd4"
server_key_version: "encryptionkey@0"
I20260806 15:05:52.494684 23577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "b87064a86f4b49fe85d745118a59df24"
format_stamp: "Formatted at 2026-08-06 15:05:52 on dist-test-slave-tc2s"
server_key: "b1875e549dcda48f5344a81eed1dbc1e"
server_key_iv: "38cad7a2209da84bd4af2e6f2a352fd4"
server_key_version: "encryptionkey@0"
I20260806 15:05:52.495857 23577 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:05:52.496500 23601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:52.496659 23577 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:05:52.496744 23577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "b87064a86f4b49fe85d745118a59df24"
format_stamp: "Formatted at 2026-08-06 15:05:52 on dist-test-slave-tc2s"
server_key: "b1875e549dcda48f5344a81eed1dbc1e"
server_key_iv: "38cad7a2209da84bd4af2e6f2a352fd4"
server_key_version: "encryptionkey@0"
I20260806 15:05:52.496803 23577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:52.507507 23577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:52.509110 23577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:52.509233 23577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:52.513067 23577 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:39435
I20260806 15:05:52.513100 23675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:39435 every 8 connection(s)
I20260806 15:05:52.513394 23577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:05:52.514240 23680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:05:52.516376 23680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24: Bootstrap starting.
I20260806 15:05:52.516899 23680 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24: Neither blocks nor log segments found. Creating new log.
I20260806 15:05:52.517158 23680 log.cc:824] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24: Log is configured to *not* fsync() on all Append() calls
I20260806 15:05:52.517742 23680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24: No bootstrap required, opened a new log
I20260806 15:05:52.519289 23680 raft_consensus.cc:348] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87064a86f4b49fe85d745118a59df24" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39435 } }
I20260806 15:05:52.519420 23680 raft_consensus.cc:374] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:05:52.519454 23680 raft_consensus.cc:729] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b87064a86f4b49fe85d745118a59df24, State: Initialized, Role: FOLLOWER
I20260806 15:05:52.519552 23680 consensus_queue.cc:260] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87064a86f4b49fe85d745118a59df24" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39435 } }
I20260806 15:05:52.519600 23680 raft_consensus.cc:388] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:05:52.519618 23680 raft_consensus.cc:482] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:05:52.519649 23680 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:05:52.520087 23680 raft_consensus.cc:504] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87064a86f4b49fe85d745118a59df24" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39435 } }
I20260806 15:05:52.520169 23680 leader_election.cc:302] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b87064a86f4b49fe85d745118a59df24; no voters:
I20260806 15:05:52.520298 23680 leader_election.cc:288] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:05:52.520349 23685 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:05:52.520522 23680 sys_catalog.cc:564] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:05:52.520542 23685 raft_consensus.cc:686] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [term 1 LEADER]: Becoming Leader. State: Replica: b87064a86f4b49fe85d745118a59df24, State: Running, Role: LEADER
I20260806 15:05:52.520673 23685 consensus_queue.cc:237] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87064a86f4b49fe85d745118a59df24" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39435 } }
I20260806 15:05:52.521040 23688 sys_catalog.cc:455] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b87064a86f4b49fe85d745118a59df24. Latest consensus state: current_term: 1 leader_uuid: "b87064a86f4b49fe85d745118a59df24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87064a86f4b49fe85d745118a59df24" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39435 } } }
I20260806 15:05:52.521204 23688 sys_catalog.cc:458] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [sys.catalog]: This master's current role is: LEADER
I20260806 15:05:52.521317 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 23577
I20260806 15:05:52.521420 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:05:52.521605 945 external_mini_cluster.cc:1428] Setting key 9bad747eb7e78ea5796e8234c7379634
I20260806 15:05:52.521649 23700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:05:52.521083 23687 sys_catalog.cc:455] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b87064a86f4b49fe85d745118a59df24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87064a86f4b49fe85d745118a59df24" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39435 } } }
I20260806 15:05:52.521847 23687 sys_catalog.cc:458] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [sys.catalog]: This master's current role is: LEADER
I20260806 15:05:52.522186 23700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:05:52.523810 23700 catalog_manager.cc:1347] Generated new cluster ID: 9f18e9a24e384ccabec49e161db8d837
I20260806 15:05:52.523864 23700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 15:05:52 dist-test-slave-tc2s krb5kdc[23562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:05:52.531445 23717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:52.522159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50424 (local address 127.0.236.126:39435)
0806 15:05:52.522595 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:52.522602 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:52.522625 (+ 23us) server_negotiation.cc:407] Connection header received
0806 15:05:52.522793 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:52.522798 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:52.522850 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:52.523692 (+ 842us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:52.523783 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:52.524693 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:52.525420 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:52.525841 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:52.525971 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:52.528060 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:52.528064 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:52.528065 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:52.528076 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:52.530075 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:52.530384 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:52.530387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:52.530388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:52.530472 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:52.530644 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:52.530645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:52.530645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:52.530878 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:52.531011 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:52.531113 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 15:05:52.531241 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":70,"threads_started":1}
I20260806 15:05:52.539265 23700 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:05:52.539721 23700 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:05:52.551419 23700 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24: Generated new TSK 0
I20260806 15:05:52.551574 23700 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.236.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
I20260806 15:05:52.581918 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:0
--local_ip_for_outbound_sockets=127.0.236.65
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:39435
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:05:52.661158 23723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:52.661348 23723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:52.661369 23723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:52.662454 23723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:52.662488 23723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:52.662513 23723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:05:52.663862 23723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:39435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:52.664052 23723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:52.664216 23723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:05:52.668834 23737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:52.668854 23723 server_base.cc:1034] running on GCE node
W20260806 15:05:52.668838 23735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:52.668838 23734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:52.669219 23723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:52.669435 23723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:52.670565 23723 hybrid_clock.cc:648] HybridClock initialized: now 1786028752670557 us; error 27 us; skew 500 ppm
Aug 06 15:05:52 dist-test-slave-tc2s krb5kdc[23562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028752, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:52.672899 23723 init.cc:377] Logged in from keytab as kudu/127.0.236.65@KRBTEST.COM (short username kudu)
I20260806 15:05:52.673259 23723 webserver.cc:460] Webserver started at http://127.0.236.65:45135/ using document root <none> and password file <none>
I20260806 15:05:52.673444 23723 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:52.673482 23723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:05:52.673568 23723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:05:52.674520 23723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/data/instance:
uuid: "c4f9b8139a434eada46943ee89d6cc01"
format_stamp: "Formatted at 2026-08-06 15:05:52 on dist-test-slave-tc2s"
server_key: "cd4bc9e116c03dcb18a43f8029e62899"
server_key_iv: "9f899b0857a6cf3522c436f93677310b"
server_key_version: "encryptionkey@0"
I20260806 15:05:52.674798 23723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/wal/instance:
uuid: "c4f9b8139a434eada46943ee89d6cc01"
format_stamp: "Formatted at 2026-08-06 15:05:52 on dist-test-slave-tc2s"
server_key: "cd4bc9e116c03dcb18a43f8029e62899"
server_key_iv: "9f899b0857a6cf3522c436f93677310b"
server_key_version: "encryptionkey@0"
I20260806 15:05:52.675953 23723 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:05:52.676560 23752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:52.676704 23723 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:05:52.676779 23723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "c4f9b8139a434eada46943ee89d6cc01"
format_stamp: "Formatted at 2026-08-06 15:05:52 on dist-test-slave-tc2s"
server_key: "cd4bc9e116c03dcb18a43f8029e62899"
server_key_iv: "9f899b0857a6cf3522c436f93677310b"
server_key_version: "encryptionkey@0"
I20260806 15:05:52.676841 23723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:52.686322 23723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:52.688230 23723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:52.688349 23723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:52.688536 23723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:05:52.688791 23723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:05:52.688823 23723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:52.688843 23723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:05:52.688867 23723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:52.694347 23723 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:40075
I20260806 15:05:52.694386 23941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:40075 every 8 connection(s)
I20260806 15:05:52.694669 23723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:05:52.696246 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 23723
I20260806 15:05:52.696332 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/wal/instance
I20260806 15:05:52.696517 945 external_mini_cluster.cc:1428] Setting key e761e3cb3cea17e1328e15aa03cc02b3
Aug 06 15:05:52 dist-test-slave-tc2s krb5kdc[23562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028752, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:05:52.703490 23717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:52.695270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:50339 (local address 127.0.236.126:39435)
0806 15:05:52.695345 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:52.695347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:52.695483 (+ 136us) server_negotiation.cc:407] Connection header received
0806 15:05:52.695888 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:52.695889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:52.695904 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:52.695953 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:52.696925 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:52.697601 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:52.698734 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:52.698944 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:52.698976 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:52.700987 (+ 2011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:52.700989 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:52.700989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:52.700996 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:52.702276 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:52.702638 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:52.702639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:52.702640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:52.702695 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:52.702901 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:52.702902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:52.702902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:52.703017 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:52.703072 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:52.703320 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 15:05:52.703405 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 15:05:52.703680 23945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:52.695236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.65:50339)
0806 15:05:52.695377 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:52.695415 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:52.695822 (+ 407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:52.696000 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:52.696003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:52.696595 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:52.696854 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:52.696859 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:52.697677 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:52.697678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:52.698634 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:52.698637 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:52.699044 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:52.699044 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:52.699138 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:52.699580 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:52.699587 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:52.700901 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:52.702348 (+ 1447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:52.702356 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:52.702358 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:52.702573 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:52.702747 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:52.702747 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:52.702748 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:52.702847 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:52.703072 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:52.703079 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:52.703205 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 15:05:52.703288 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 15:05:52.703440 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":82,"threads_started":1}
I20260806 15:05:52.704128 23942 heartbeater.cc:347] Connected to a master server at 127.0.236.126:39435
I20260806 15:05:52.704236 23942 heartbeater.cc:464] Registering TS with master...
I20260806 15:05:52.704423 23942 heartbeater.cc:511] Master 127.0.236.126:39435 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.0.236.66@KRBTEST.COM; defaulting to no policy
I20260806 15:05:52.704923 23623 ts_manager.cc:194] Registered new tserver with Master: c4f9b8139a434eada46943ee89d6cc01 (127.0.236.65:40075)
I20260806 15:05:52.705526 23623 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.65@KRBTEST.COM'} at 127.0.236.65:50339
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
I20260806 15:05:52.720152 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:0
--local_ip_for_outbound_sockets=127.0.236.66
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:39435
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:05:52.798834 23951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:52.799011 23951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:52.799033 23951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:52.800150 23951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:52.800201 23951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:52.800230 23951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:05:52.801654 23951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:39435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:52.801826 23951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:52.802011 23951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:05:52.806533 23961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:52.806538 23959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:52.806619 23951 server_base.cc:1034] running on GCE node
W20260806 15:05:52.806541 23963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:52.806962 23951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:52.807186 23951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:52.808326 23951 hybrid_clock.cc:648] HybridClock initialized: now 1786028752808314 us; error 29 us; skew 500 ppm
Aug 06 15:05:52 dist-test-slave-tc2s krb5kdc[23562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028752, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:52.810617 23951 init.cc:377] Logged in from keytab as kudu/127.0.236.66@KRBTEST.COM (short username kudu)
I20260806 15:05:52.811002 23951 webserver.cc:460] Webserver started at http://127.0.236.66:32853/ using document root <none> and password file <none>
I20260806 15:05:52.811205 23951 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:52.811247 23951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:05:52.811367 23951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:05:52.812299 23951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/data/instance:
uuid: "9f2c60fb4db442c1b269ed12633570ee"
format_stamp: "Formatted at 2026-08-06 15:05:52 on dist-test-slave-tc2s"
server_key: "df6ebcfd7b4a59dd32d829454bad19ed"
server_key_iv: "98419b7e630623038e7b68c96bebf0bc"
server_key_version: "encryptionkey@0"
I20260806 15:05:52.812613 23951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/wal/instance:
uuid: "9f2c60fb4db442c1b269ed12633570ee"
format_stamp: "Formatted at 2026-08-06 15:05:52 on dist-test-slave-tc2s"
server_key: "df6ebcfd7b4a59dd32d829454bad19ed"
server_key_iv: "98419b7e630623038e7b68c96bebf0bc"
server_key_version: "encryptionkey@0"
I20260806 15:05:52.813750 23951 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:05:52.814445 23980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:52.814572 23951 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:52.814639 23951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "9f2c60fb4db442c1b269ed12633570ee"
format_stamp: "Formatted at 2026-08-06 15:05:52 on dist-test-slave-tc2s"
server_key: "df6ebcfd7b4a59dd32d829454bad19ed"
server_key_iv: "98419b7e630623038e7b68c96bebf0bc"
server_key_version: "encryptionkey@0"
I20260806 15:05:52.814702 23951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:52.824802 23951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:52.826378 23951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:52.826503 23951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:52.826690 23951 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:05:52.826948 23951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:05:52.826980 23951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:52.826999 23951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:05:52.827023 23951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:52.832271 23951 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:37857
I20260806 15:05:52.832333 24164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:37857 every 8 connection(s)
I20260806 15:05:52.832578 23951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
I20260806 15:05:52.834350 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 23951
I20260806 15:05:52.834444 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/wal/instance
I20260806 15:05:52.834617 945 external_mini_cluster.cc:1428] Setting key f54496d7516073f718f2036f618733c7
Aug 06 15:05:52 dist-test-slave-tc2s krb5kdc[23562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028752, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:05:52.841521 23717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:52.833182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:38217 (local address 127.0.236.126:39435)
0806 15:05:52.833228 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:52.833230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:52.833391 (+ 161us) server_negotiation.cc:407] Connection header received
0806 15:05:52.833851 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:52.833852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:52.833866 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:52.833949 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:52.834890 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:52.835542 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:52.836479 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:52.836746 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:52.836816 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:52.839007 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:52.839008 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:52.839008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:52.839015 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:52.840228 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:52.840677 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:52.840678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:52.840678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:52.840723 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:52.840972 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:52.840973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:52.840973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:52.841062 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:52.841104 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:52.841389 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 15:05:52.841458 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:05:52.841744 24169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:52.833142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.66:38217)
0806 15:05:52.833279 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:52.833321 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:52.833754 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:52.834016 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:52.834020 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:52.834580 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:52.834799 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:52.834804 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:52.835632 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:52.835633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:52.836390 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:52.836392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:52.836839 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:52.836840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:52.836991 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:52.837397 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:52.837407 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:52.838902 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:52.840305 (+ 1403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:52.840313 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:52.840315 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:52.840609 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:52.840774 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:52.840775 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:52.840775 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:52.840907 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:52.841113 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:52.841119 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:52.841272 (+ 153us) client_negotiation.cc:769] Sending connection context
0806 15:05:52.841381 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 15:05:52.841537 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
I20260806 15:05:52.842211 24165 heartbeater.cc:347] Connected to a master server at 127.0.236.126:39435
I20260806 15:05:52.842301 24165 heartbeater.cc:464] Registering TS with master...
I20260806 15:05:52.842463 24165 heartbeater.cc:511] Master 127.0.236.126:39435 requested a full tablet report, sending...
I20260806 15:05:52.842834 23623 ts_manager.cc:194] Registered new tserver with Master: 9f2c60fb4db442c1b269ed12633570ee (127.0.236.66:37857)
I20260806 15:05:52.843187 23623 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.66@KRBTEST.COM'} at 127.0.236.66:38217
WARNING: no policy specified for kudu/127.0.236.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
I20260806 15:05:52.857765 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:0
--local_ip_for_outbound_sockets=127.0.236.67
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:39435
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:05:52.935575 24172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:05:52.935760 24172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:05:52.935791 24172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:05:52.936861 24172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:05:52.936901 24172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:05:52.936935 24172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:05:52.938328 24172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:39435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:05:52.938542 24172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:05:52.938736 24172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:05:52.943286 24180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:52.943303 24181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:05:52.943303 24183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:05:52.943572 24172 server_base.cc:1034] running on GCE node
I20260806 15:05:52.943769 24172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:05:52.943980 24172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:05:52.945088 24172 hybrid_clock.cc:648] HybridClock initialized: now 1786028752945075 us; error 26 us; skew 500 ppm
Aug 06 15:05:52 dist-test-slave-tc2s krb5kdc[23562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028752, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:05:52.947372 24172 init.cc:377] Logged in from keytab as kudu/127.0.236.67@KRBTEST.COM (short username kudu)
I20260806 15:05:52.947757 24172 webserver.cc:460] Webserver started at http://127.0.236.67:43843/ using document root <none> and password file <none>
I20260806 15:05:52.947950 24172 fs_manager.cc:359] Metadata directory not provided
I20260806 15:05:52.947996 24172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:05:52.948120 24172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:05:52.949065 24172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/data/instance:
uuid: "ea5947f4e19b4c6197aa84d31b44616c"
format_stamp: "Formatted at 2026-08-06 15:05:52 on dist-test-slave-tc2s"
server_key: "ac1c95289126702594a3afcfa5a2b565"
server_key_iv: "90a63a2e6ca10e22f08337c3c1f758cb"
server_key_version: "encryptionkey@0"
I20260806 15:05:52.949375 24172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/wal/instance:
uuid: "ea5947f4e19b4c6197aa84d31b44616c"
format_stamp: "Formatted at 2026-08-06 15:05:52 on dist-test-slave-tc2s"
server_key: "ac1c95289126702594a3afcfa5a2b565"
server_key_iv: "90a63a2e6ca10e22f08337c3c1f758cb"
server_key_version: "encryptionkey@0"
I20260806 15:05:52.950659 24172 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:05:52.951272 24197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:52.951439 24172 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:05:52.951521 24172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "ea5947f4e19b4c6197aa84d31b44616c"
format_stamp: "Formatted at 2026-08-06 15:05:52 on dist-test-slave-tc2s"
server_key: "ac1c95289126702594a3afcfa5a2b565"
server_key_iv: "90a63a2e6ca10e22f08337c3c1f758cb"
server_key_version: "encryptionkey@0"
I20260806 15:05:52.951586 24172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:05:52.975520 24172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:05:52.977448 24172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:05:52.977576 24172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:05:52.977766 24172 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:05:52.978040 24172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:05:52.978075 24172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:52.978092 24172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:05:52.978116 24172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:05:52.983562 24172 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:37451
I20260806 15:05:52.983644 24389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:37451 every 8 connection(s)
I20260806 15:05:52.983872 24172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
Aug 06 15:05:52 dist-test-slave-tc2s krb5kdc[23562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028752, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:05:52.991739 23717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:52.984468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:47217 (local address 127.0.236.126:39435)
0806 15:05:52.984513 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:52.984515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:52.984686 (+ 171us) server_negotiation.cc:407] Connection header received
0806 15:05:52.985125 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:52.985126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:52.985135 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:52.985181 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:52.986093 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:52.986588 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:52.987483 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:52.987658 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:52.987701 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:52.989430 (+ 1729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:52.989432 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:52.989432 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:52.989439 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:52.990682 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:52.990982 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:52.990983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:52.990983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:52.991030 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:52.991277 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:52.991278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:52.991278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:52.991369 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:52.991412 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:52.991623 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 15:05:52.991677 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 15:05:52.991940 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:52.984423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.67:47217)
0806 15:05:52.984583 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:52.984620 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:52.985050 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:52.985234 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:52.985237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:52.985756 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:05:52.986005 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:52.986011 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:52.986652 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:52.986653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:52.987421 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:52.987422 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:52.987708 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:52.987708 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:52.987798 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:52.988138 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:05:52.988145 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:05:52.989347 (+ 1202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:05:52.990739 (+ 1392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:52.990743 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:52.990744 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:52.990930 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:52.991111 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:05:52.991112 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:05:52.991113 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:05:52.991222 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:05:52.991416 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:05:52.991419 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:05:52.991531 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 15:05:52.991605 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 15:05:52.991730 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
I20260806 15:05:52.991906 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 24172
I20260806 15:05:52.992025 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/wal/instance
I20260806 15:05:52.992213 945 external_mini_cluster.cc:1428] Setting key 8636bf02bb0c5a0fbe8985e58f889f4f
I20260806 15:05:52.992466 24392 heartbeater.cc:347] Connected to a master server at 127.0.236.126:39435
I20260806 15:05:52.992559 24392 heartbeater.cc:464] Registering TS with master...
I20260806 15:05:52.992736 24392 heartbeater.cc:511] Master 127.0.236.126:39435 requested a full tablet report, sending...
I20260806 15:05:52.993095 23623 ts_manager.cc:194] Registered new tserver with Master: ea5947f4e19b4c6197aa84d31b44616c (127.0.236.67:37451)
I20260806 15:05:52.993449 23623 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.67@KRBTEST.COM'} at 127.0.236.67:47217
I20260806 15:05:53.003768 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:05:53.010283 23717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:53.005027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38706 (local address 127.0.236.126:39435)
0806 15:05:53.005068 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:53.005069 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:53.005170 (+ 101us) server_negotiation.cc:407] Connection header received
0806 15:05:53.005232 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:53.005232 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:53.005242 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:53.005288 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:05:53.005812 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:53.006330 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:53.007045 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:53.007237 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:53.007288 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:53.008257 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:05:53.008259 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:05:53.008259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:05:53.008267 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:05:53.009227 (+ 960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:53.009526 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:53.009527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:53.009528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:53.009580 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:05:53.009735 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:05:53.009736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:05:53.009736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:05:53.009873 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:05:53.010015 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:53.010125 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 15:05:53.010200 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:05:53.011508 23623 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:05:53.012924 23623 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:05:53.018989 24409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:53.014866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33636 (local address 127.0.236.65:40075)
0806 15:05:53.015461 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:53.015465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:53.015474 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:05:53.015512 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:53.015514 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:53.015529 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:53.015599 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:05:53.015797 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:53.016776 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:53.017837 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:53.018592 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:53.018666 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:53.018721 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:53.018870 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 15:05:53.018914 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":96,"threads_started":1}
I20260806 15:05:53.018998 24405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:53.014737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:40075 (local address 127.0.0.1:33636)
0806 15:05:53.015062 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:53.015099 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:53.015158 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:53.015679 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:53.015680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:53.015682 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:05:53.015723 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:53.015724 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:53.016863 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:53.016864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:53.017751 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:53.017753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:53.018669 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:53.018671 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:53.018761 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:53.018776 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 15:05:53.018848 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 15:05:53.018910 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":4,"thread_start_us":50,"threads_started":1}
I20260806 15:05:53.018998 24411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:53.014922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:37451 (local address 127.0.0.1:55718)
0806 15:05:53.015280 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:53.015289 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:53.015335 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:53.015577 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:53.015579 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:53.015581 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:05:53.015615 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:53.015617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:53.016456 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:53.016457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:53.017767 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:53.017770 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:53.018671 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:53.018672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:53.018761 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:53.018776 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 15:05:53.018849 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 15:05:53.018912 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":32,"thread_start_us":79,"threads_started":1}
I20260806 15:05:53.019719 24407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:53.014757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:37857 (local address 127.0.0.1:55760)
0806 15:05:53.015066 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:53.015102 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:53.015170 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:53.015349 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:53.015352 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:53.015354 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:05:53.015417 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:53.015419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:53.016617 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:53.016618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:53.017768 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:53.017770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:53.019527 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:53.019528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:53.019592 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:53.019596 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:05:53.019644 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 15:05:53.019673 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":6,"thread_start_us":60,"threads_started":1}
I20260806 15:05:53.019760 23830 tablet_service.cc:1467] Processing CreateTablet for tablet f26e2d9a473a44f780d8b70c84ca0573 (DEFAULT_TABLE table=test-table [id=b25f5472fa874cd9acc0c764a3c6830e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:05:53.019766 24410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:53.014906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55760 (local address 127.0.236.66:37857)
0806 15:05:53.015059 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:53.015063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:53.015174 (+ 111us) server_negotiation.cc:407] Connection header received
0806 15:05:53.015241 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:53.015243 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:53.015260 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:53.015337 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:05:53.015480 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:53.016522 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:53.018756 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:53.019447 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:53.019518 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:53.019570 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:53.019656 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 15:05:53.019695 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":74,"threads_started":1}
I20260806 15:05:53.020115 23830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f26e2d9a473a44f780d8b70c84ca0573. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:05:53.020609 24058 tablet_service.cc:1467] Processing CreateTablet for tablet f26e2d9a473a44f780d8b70c84ca0573 (DEFAULT_TABLE table=test-table [id=b25f5472fa874cd9acc0c764a3c6830e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:05:53.020898 24058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f26e2d9a473a44f780d8b70c84ca0573. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:05:53.019287 24414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:53.015068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55718 (local address 127.0.236.67:37451)
0806 15:05:53.015238 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:53.015242 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:53.015339 (+ 97us) server_negotiation.cc:407] Connection header received
0806 15:05:53.015393 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:53.015395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:53.015411 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:53.015494 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:05:53.015685 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:53.016359 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:53.017837 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:53.018592 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:53.018666 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:53.018721 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:53.019182 (+ 461us) server_negotiation.cc:299] Negotiation successful
0806 15:05:53.019220 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":84,"threads_started":1}
I20260806 15:05:53.022588 24284 tablet_service.cc:1467] Processing CreateTablet for tablet f26e2d9a473a44f780d8b70c84ca0573 (DEFAULT_TABLE table=test-table [id=b25f5472fa874cd9acc0c764a3c6830e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:05:53.023092 24284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f26e2d9a473a44f780d8b70c84ca0573. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:05:53.023772 24416 tablet_bootstrap.cc:492] T f26e2d9a473a44f780d8b70c84ca0573 P 9f2c60fb4db442c1b269ed12633570ee: Bootstrap starting.
I20260806 15:05:53.024427 24416 tablet_bootstrap.cc:654] T f26e2d9a473a44f780d8b70c84ca0573 P 9f2c60fb4db442c1b269ed12633570ee: Neither blocks nor log segments found. Creating new log.
I20260806 15:05:53.024963 24416 log.cc:824] T f26e2d9a473a44f780d8b70c84ca0573 P 9f2c60fb4db442c1b269ed12633570ee: Log is configured to *not* fsync() on all Append() calls
I20260806 15:05:53.025702 24416 tablet_bootstrap.cc:492] T f26e2d9a473a44f780d8b70c84ca0573 P 9f2c60fb4db442c1b269ed12633570ee: No bootstrap required, opened a new log
I20260806 15:05:53.025779 24416 ts_tablet_manager.cc:1397] T f26e2d9a473a44f780d8b70c84ca0573 P 9f2c60fb4db442c1b269ed12633570ee: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:05:53.025955 24424 tablet_bootstrap.cc:492] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c: Bootstrap starting.
I20260806 15:05:53.026116 24415 tablet_bootstrap.cc:492] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01: Bootstrap starting.
I20260806 15:05:53.026646 24424 tablet_bootstrap.cc:654] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c: Neither blocks nor log segments found. Creating new log.
I20260806 15:05:53.026773 24415 tablet_bootstrap.cc:654] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01: Neither blocks nor log segments found. Creating new log.
I20260806 15:05:53.027099 24424 log.cc:824] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c: Log is configured to *not* fsync() on all Append() calls
I20260806 15:05:53.027338 24415 log.cc:824] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01: Log is configured to *not* fsync() on all Append() calls
I20260806 15:05:53.027752 24424 tablet_bootstrap.cc:492] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c: No bootstrap required, opened a new log
I20260806 15:05:53.027678 24416 raft_consensus.cc:348] T f26e2d9a473a44f780d8b70c84ca0573 P 9f2c60fb4db442c1b269ed12633570ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } } peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } }
I20260806 15:05:53.027822 24424 ts_tablet_manager.cc:1397] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:05:53.027818 24416 raft_consensus.cc:374] T f26e2d9a473a44f780d8b70c84ca0573 P 9f2c60fb4db442c1b269ed12633570ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:05:53.027854 24416 raft_consensus.cc:729] T f26e2d9a473a44f780d8b70c84ca0573 P 9f2c60fb4db442c1b269ed12633570ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f2c60fb4db442c1b269ed12633570ee, State: Initialized, Role: FOLLOWER
I20260806 15:05:53.027998 24416 consensus_queue.cc:260] T f26e2d9a473a44f780d8b70c84ca0573 P 9f2c60fb4db442c1b269ed12633570ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } } peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } }
I20260806 15:05:53.028066 24415 tablet_bootstrap.cc:492] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01: No bootstrap required, opened a new log
I20260806 15:05:53.028117 24415 ts_tablet_manager.cc:1397] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:05:53.028275 24416 ts_tablet_manager.cc:1428] T f26e2d9a473a44f780d8b70c84ca0573 P 9f2c60fb4db442c1b269ed12633570ee: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 15:05:53.028414 24165 heartbeater.cc:503] Master 127.0.236.126:39435 was elected leader, sending a full tablet report...
I20260806 15:05:53.029448 24415 raft_consensus.cc:348] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } } peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } }
I20260806 15:05:53.029570 24415 raft_consensus.cc:374] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:05:53.029592 24415 raft_consensus.cc:729] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4f9b8139a434eada46943ee89d6cc01, State: Initialized, Role: FOLLOWER
I20260806 15:05:53.029500 24424 raft_consensus.cc:348] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } } peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } }
I20260806 15:05:53.029645 24424 raft_consensus.cc:374] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:05:53.029680 24424 raft_consensus.cc:729] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea5947f4e19b4c6197aa84d31b44616c, State: Initialized, Role: FOLLOWER
I20260806 15:05:53.029695 24415 consensus_queue.cc:260] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } } peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } }
I20260806 15:05:53.029785 24424 consensus_queue.cc:260] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } } peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } }
I20260806 15:05:53.029984 23942 heartbeater.cc:503] Master 127.0.236.126:39435 was elected leader, sending a full tablet report...
I20260806 15:05:53.030071 24424 ts_tablet_manager.cc:1428] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260806 15:05:53.030190 24392 heartbeater.cc:503] Master 127.0.236.126:39435 was elected leader, sending a full tablet report...
I20260806 15:05:53.029978 24415 ts_tablet_manager.cc:1428] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:05:53.045001 24430 raft_consensus.cc:482] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:05:53.045078 24430 raft_consensus.cc:504] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } } peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } }
I20260806 15:05:53.045336 24430 leader_election.cc:288] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4f9b8139a434eada46943ee89d6cc01 (127.0.236.65:40075), 9f2c60fb4db442c1b269ed12633570ee (127.0.236.66:37857)
I20260806 15:05:53.047734 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:53.045458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:40075 (local address 127.0.236.67:44485)
0806 15:05:53.045501 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:53.045510 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:53.045544 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:53.045649 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:53.045650 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:53.045651 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:05:53.045702 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:53.045703 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:53.046312 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:53.046312 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:53.047021 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:53.047023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:53.047522 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:53.047523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:53.047590 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:53.047593 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:05:53.047635 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 15:05:53.047670 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":28}
I20260806 15:05:53.047734 24409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:53.045534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:44485 (local address 127.0.236.65:40075)
0806 15:05:53.045567 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:53.045569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:53.045576 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:05:53.045595 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:53.045596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:53.045603 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:53.045648 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:05:53.045753 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:53.046240 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:53.047095 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:53.047458 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:53.047519 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:53.047526 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:53.047646 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 15:05:53.047678 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 15:05:53.047863 24410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:53.045514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:57065 (local address 127.0.236.66:37857)
0806 15:05:53.045553 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:53.045555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:53.045663 (+ 108us) server_negotiation.cc:407] Connection header received
0806 15:05:53.045715 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:53.045716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:53.045724 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:53.045763 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:05:53.045888 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:53.046330 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:53.047208 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:53.047567 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:53.047626 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:53.047634 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:53.047764 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 15:05:53.047798 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 15:05:53.048040 24431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:53.045467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:37857 (local address 127.0.236.67:57065)
0806 15:05:53.045598 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:53.045607 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:05:53.045662 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:05:53.045771 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:05:53.045771 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:05:53.045773 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:05:53.045837 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:53.045838 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:53.046399 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:53.046400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:53.047133 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:05:53.047135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:05:53.047637 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:05:53.047638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:05:53.047707 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:53.047711 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:05:53.047755 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 15:05:53.047966 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
I20260806 15:05:53.048015 23859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f26e2d9a473a44f780d8b70c84ca0573" candidate_uuid: "ea5947f4e19b4c6197aa84d31b44616c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4f9b8139a434eada46943ee89d6cc01" is_pre_election: true
I20260806 15:05:53.048110 23859 raft_consensus.cc:2454] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea5947f4e19b4c6197aa84d31b44616c in term 0.
I20260806 15:05:53.048281 24203 leader_election.cc:302] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4f9b8139a434eada46943ee89d6cc01, ea5947f4e19b4c6197aa84d31b44616c; no voters:
I20260806 15:05:53.048314 24094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f26e2d9a473a44f780d8b70c84ca0573" candidate_uuid: "ea5947f4e19b4c6197aa84d31b44616c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f2c60fb4db442c1b269ed12633570ee" is_pre_election: true
I20260806 15:05:53.048394 24430 raft_consensus.cc:2781] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:05:53.048426 24094 raft_consensus.cc:2454] T f26e2d9a473a44f780d8b70c84ca0573 P 9f2c60fb4db442c1b269ed12633570ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea5947f4e19b4c6197aa84d31b44616c in term 0.
I20260806 15:05:53.048453 24430 raft_consensus.cc:482] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:05:53.048480 24430 raft_consensus.cc:3037] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:05:53.049039 24430 raft_consensus.cc:504] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } } peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } }
I20260806 15:05:53.049146 24430 leader_election.cc:288] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c [CANDIDATE]: Term 1 election: Requested vote from peers c4f9b8139a434eada46943ee89d6cc01 (127.0.236.65:40075), 9f2c60fb4db442c1b269ed12633570ee (127.0.236.66:37857)
I20260806 15:05:53.049284 23859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f26e2d9a473a44f780d8b70c84ca0573" candidate_uuid: "ea5947f4e19b4c6197aa84d31b44616c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4f9b8139a434eada46943ee89d6cc01"
I20260806 15:05:53.049341 24094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f26e2d9a473a44f780d8b70c84ca0573" candidate_uuid: "ea5947f4e19b4c6197aa84d31b44616c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f2c60fb4db442c1b269ed12633570ee"
I20260806 15:05:53.049350 23859 raft_consensus.cc:3037] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:05:53.049417 24094 raft_consensus.cc:3037] T f26e2d9a473a44f780d8b70c84ca0573 P 9f2c60fb4db442c1b269ed12633570ee [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:05:53.050107 23859 raft_consensus.cc:2454] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea5947f4e19b4c6197aa84d31b44616c in term 1.
I20260806 15:05:53.050158 24094 raft_consensus.cc:2454] T f26e2d9a473a44f780d8b70c84ca0573 P 9f2c60fb4db442c1b269ed12633570ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea5947f4e19b4c6197aa84d31b44616c in term 1.
I20260806 15:05:53.050287 24203 leader_election.cc:302] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4f9b8139a434eada46943ee89d6cc01, ea5947f4e19b4c6197aa84d31b44616c; no voters:
I20260806 15:05:53.050396 24430 raft_consensus.cc:2781] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:05:53.050525 24430 raft_consensus.cc:686] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c [term 1 LEADER]: Becoming Leader. State: Replica: ea5947f4e19b4c6197aa84d31b44616c, State: Running, Role: LEADER
I20260806 15:05:53.050601 24430 consensus_queue.cc:237] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } } peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } }
I20260806 15:05:53.051246 23622 catalog_manager.cc:5495] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c reported cstate change: term changed from 0 to 1, leader changed from <none> to ea5947f4e19b4c6197aa84d31b44616c (127.0.236.67). New cstate: current_term: 1 leader_uuid: "ea5947f4e19b4c6197aa84d31b44616c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260806 15:05:53.063000 24414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:53.060798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55728 (local address 127.0.236.67:37451)
0806 15:05:53.060847 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 15:05:53.060848 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:05:53.060872 (+ 24us) server_negotiation.cc:407] Connection header received
0806 15:05:53.060899 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:05:53.060900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:05:53.060915 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:05:53.060971 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:05:53.061084 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:53.061515 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:53.062304 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:05:53.062498 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:05:53.062548 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:05:53.062696 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:05:53.062783 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:05:53.062868 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 15:05:53.062922 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 15:05:53.062939 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 15:05:53.064499 24094 raft_consensus.cc:1261] T f26e2d9a473a44f780d8b70c84ca0573 P 9f2c60fb4db442c1b269ed12633570ee [term 1 FOLLOWER]: Refusing update from remote peer ea5947f4e19b4c6197aa84d31b44616c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:05:53.064509 23859 raft_consensus.cc:1261] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Refusing update from remote peer ea5947f4e19b4c6197aa84d31b44616c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:05:53.064775 24433 consensus_queue.cc:1035] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:05:53.064885 24430 consensus_queue.cc:1035] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:05:53.067013 24436 mvcc.cc:205] Tried to move back new op lower bound from 7315573772550086656 to 7315573772495831040. Current Snapshot: MvccSnapshot[applied={T|T < 7315573772550086656}]
I20260806 15:05:53.067358 24438 mvcc.cc:205] Tried to move back new op lower bound from 7315573772550086656 to 7315573772495831040. Current Snapshot: MvccSnapshot[applied={T|T < 7315573772550086656}]
I20260806 15:05:53.067623 24440 mvcc.cc:205] Tried to move back new op lower bound from 7315573772550086656 to 7315573772495831040. Current Snapshot: MvccSnapshot[applied={T|T < 7315573772550086656}]
I20260806 15:05:53.069242 23623 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:05:53.069314 23623 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:05:53.069932 23623 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24: Sending DeleteTablet for 3 replicas of tablet f26e2d9a473a44f780d8b70c84ca0573
I20260806 15:05:53.070220 23830 tablet_service.cc:1514] Processing DeleteTablet for tablet f26e2d9a473a44f780d8b70c84ca0573 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:05:53 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:33636
I20260806 15:05:53.070268 24058 tablet_service.cc:1514] Processing DeleteTablet for tablet f26e2d9a473a44f780d8b70c84ca0573 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:05:53 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:55760
I20260806 15:05:53.070266 24284 tablet_service.cc:1514] Processing DeleteTablet for tablet f26e2d9a473a44f780d8b70c84ca0573 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:05:53 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:55718
I20260806 15:05:53.070453 24452 tablet_replica.cc:330] T f26e2d9a473a44f780d8b70c84ca0573 P 9f2c60fb4db442c1b269ed12633570ee: stopping tablet replica
I20260806 15:05:53.070453 24451 tablet_replica.cc:330] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01: stopping tablet replica
I20260806 15:05:53.070545 24452 raft_consensus.cc:2229] T f26e2d9a473a44f780d8b70c84ca0573 P 9f2c60fb4db442c1b269ed12633570ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:05:53.070551 24451 raft_consensus.cc:2229] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:05:53.070582 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 23723
I20260806 15:05:53.070647 24452 raft_consensus.cc:2258] T f26e2d9a473a44f780d8b70c84ca0573 P 9f2c60fb4db442c1b269ed12633570ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:05:53.071009 24452 ts_tablet_manager.cc:1905] T f26e2d9a473a44f780d8b70c84ca0573 P 9f2c60fb4db442c1b269ed12633570ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:05:53.075382 24452 ts_tablet_manager.cc:1918] T f26e2d9a473a44f780d8b70c84ca0573 P 9f2c60fb4db442c1b269ed12633570ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:05:53.075443 24452 log.cc:1196] T f26e2d9a473a44f780d8b70c84ca0573 P 9f2c60fb4db442c1b269ed12633570ee: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/wal/wals/f26e2d9a473a44f780d8b70c84ca0573
I20260806 15:05:53.075649 24452 ts_tablet_manager.cc:1939] T f26e2d9a473a44f780d8b70c84ca0573 P 9f2c60fb4db442c1b269ed12633570ee: Deleting consensus metadata
I20260806 15:05:53.075953 23602 catalog_manager.cc:4841] TS 9f2c60fb4db442c1b269ed12633570ee (127.0.236.66:37857): tablet f26e2d9a473a44f780d8b70c84ca0573 (table test-table [id=b25f5472fa874cd9acc0c764a3c6830e]) successfully deleted
I20260806 15:05:53.086166 24456 tablet_replica.cc:330] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c: stopping tablet replica
I20260806 15:05:53.086334 24456 raft_consensus.cc:2229] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:05:53.086551 24456 raft_consensus.cc:2258] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:05:53.087432 24456 ts_tablet_manager.cc:1905] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:05:53.088924 24456 ts_tablet_manager.cc:1918] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:05:53.088995 24456 log.cc:1196] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/f26e2d9a473a44f780d8b70c84ca0573
I20260806 15:05:53.089216 24456 ts_tablet_manager.cc:1939] T f26e2d9a473a44f780d8b70c84ca0573 P ea5947f4e19b4c6197aa84d31b44616c: Deleting consensus metadata
I20260806 15:05:53.089596 23608 catalog_manager.cc:4841] TS ea5947f4e19b4c6197aa84d31b44616c (127.0.236.67:37451): tablet f26e2d9a473a44f780d8b70c84ca0573 (table test-table [id=b25f5472fa874cd9acc0c764a3c6830e]) successfully deleted
W20260806 15:05:53.091902 23608 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.236.65:40075 (error 108)
I20260806 15:05:53.092136 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 23951
I20260806 15:05:53.094340 24407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:05:53.092270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:40075 (local address 127.0.0.1:33640)
0806 15:05:53.094260 (+ 1990us) negotiation.cc:107] Waiting for socket to connect
0806 15:05:53.094306 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.65:40075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1968}
W20260806 15:05:53.098889 23608 catalog_manager.cc:4584] TS c4f9b8139a434eada46943ee89d6cc01 (127.0.236.65:40075): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f26e2d9a473a44f780d8b70c84ca0573: Network error: Client connection negotiation failed: client connection to 127.0.236.65:40075: connect: Connection refused (error 111)
I20260806 15:05:53.102141 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 24172
I20260806 15:05:53.110149 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 23577
I20260806 15:06:13.122409 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:39435
--webserver_interface=127.0.236.126
--webserver_port=42213
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:39435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:06:13.202176 24458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:13.202365 24458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:13.202385 24458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:13.203485 24458 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 15:06:13.203522 24458 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 15:06:13.203536 24458 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 15:06:13.203547 24458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:06:13.203570 24458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:13.203594 24458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:13.203615 24458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:06:13.203631 24458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:06:13.205217 24458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:39435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:39435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=42213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:13.205454 24458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:13.205725 24458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:13.214658 24468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:13.214676 24471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:13.214730 24458 server_base.cc:1034] running on GCE node
W20260806 15:06:13.214694 24469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:13.215152 24458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:13.215410 24458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:13.216552 24458 hybrid_clock.cc:648] HybridClock initialized: now 1786028773216540 us; error 29 us; skew 500 ppm
Aug 06 15:06:13 dist-test-slave-tc2s krb5kdc[23562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028773, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:06:13.222040 24458 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:06:13.222498 24458 webserver.cc:460] Webserver started at http://127.0.236.126:42213/ using document root <none> and password file <none>
I20260806 15:06:13.222707 24458 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:13.222769 24458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:13.224025 24458 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:06:13.224608 24487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:13.224752 24458 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:06:13.224836 24458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "b87064a86f4b49fe85d745118a59df24"
format_stamp: "Formatted at 2026-08-06 15:05:52 on dist-test-slave-tc2s"
server_key: "b1875e549dcda48f5344a81eed1dbc1e"
server_key_iv: "38cad7a2209da84bd4af2e6f2a352fd4"
server_key_version: "encryptionkey@0"
I20260806 15:06:13.225188 24458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:13.233363 24458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:13.234965 24458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:13.235085 24458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:13.238878 24458 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:39435
I20260806 15:06:13.238916 24566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:39435 every 8 connection(s)
I20260806 15:06:13.239238 24458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:06:13.240581 24567 sys_catalog.cc:263] Verifying existing consensus state
I20260806 15:06:13.241158 24567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24: Bootstrap starting.
I20260806 15:06:13.247648 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 24458
I20260806 15:06:13.248001 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:40075
--local_ip_for_outbound_sockets=127.0.236.65
--tserver_master_addrs=127.0.236.126:39435
--webserver_port=45135
--webserver_interface=127.0.236.65
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:06:13.249434 24567 log.cc:824] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:13.251187 24567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 15:06:13.251448 24567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24: Bootstrap complete.
I20260806 15:06:13.253592 24567 raft_consensus.cc:348] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87064a86f4b49fe85d745118a59df24" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39435 } }
I20260806 15:06:13.253799 24567 raft_consensus.cc:729] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b87064a86f4b49fe85d745118a59df24, State: Initialized, Role: FOLLOWER
I20260806 15:06:13.253959 24567 consensus_queue.cc:260] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87064a86f4b49fe85d745118a59df24" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39435 } }
I20260806 15:06:13.254029 24567 raft_consensus.cc:388] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:06:13.254062 24567 raft_consensus.cc:482] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:06:13.254097 24567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [term 1 FOLLOWER]: Advancing to term 2
I20260806 15:06:13.255023 24567 raft_consensus.cc:504] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87064a86f4b49fe85d745118a59df24" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39435 } }
I20260806 15:06:13.255107 24567 leader_election.cc:302] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b87064a86f4b49fe85d745118a59df24; no voters:
I20260806 15:06:13.255246 24567 leader_election.cc:288] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 15:06:13.255293 24573 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 15:06:13.255453 24573 raft_consensus.cc:686] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [term 2 LEADER]: Becoming Leader. State: Replica: b87064a86f4b49fe85d745118a59df24, State: Running, Role: LEADER
I20260806 15:06:13.255507 24567 sys_catalog.cc:564] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:06:13.255569 24573 consensus_queue.cc:237] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87064a86f4b49fe85d745118a59df24" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39435 } }
I20260806 15:06:13.255947 24573 sys_catalog.cc:455] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b87064a86f4b49fe85d745118a59df24. Latest consensus state: current_term: 2 leader_uuid: "b87064a86f4b49fe85d745118a59df24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87064a86f4b49fe85d745118a59df24" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39435 } } }
I20260806 15:06:13.256037 24573 sys_catalog.cc:458] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [sys.catalog]: This master's current role is: LEADER
I20260806 15:06:13.256215 24586 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:06:13.255929 24575 sys_catalog.cc:455] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b87064a86f4b49fe85d745118a59df24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87064a86f4b49fe85d745118a59df24" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39435 } } }
I20260806 15:06:13.256392 24575 sys_catalog.cc:458] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [sys.catalog]: This master's current role is: LEADER
I20260806 15:06:13.257206 24586 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:06:13.257357 24586 catalog_manager.cc:1259] Loaded cluster ID: 9f18e9a24e384ccabec49e161db8d837
I20260806 15:06:13.257527 24586 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:06:13.258342 24586 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:06:13.258468 24586 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24: Loaded TSK: 0
I20260806 15:06:13.268457 24586 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24: Generated new TSK 1
I20260806 15:06:13.268538 24586 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 15:06:13.339345 24570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:13.339540 24570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:13.339569 24570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:13.340672 24570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:13.340711 24570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:13.340744 24570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:06:13.342190 24570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:40075
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=45135
--tserver_master_addrs=127.0.236.126:39435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:13.342399 24570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:13.342593 24570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:13.348109 24608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:13.348145 24607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:13.348367 24612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:13.348584 24570 server_base.cc:1034] running on GCE node
I20260806 15:06:13.348753 24570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:13.348973 24570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:13.350103 24570 hybrid_clock.cc:648] HybridClock initialized: now 1786028773350090 us; error 30 us; skew 500 ppm
Aug 06 15:06:13 dist-test-slave-tc2s krb5kdc[23562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028773, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:06:13.352357 24570 init.cc:377] Logged in from keytab as kudu/127.0.236.65@KRBTEST.COM (short username kudu)
I20260806 15:06:13.352757 24570 webserver.cc:460] Webserver started at http://127.0.236.65:45135/ using document root <none> and password file <none>
I20260806 15:06:13.352957 24570 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:13.353029 24570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:13.354306 24570 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:06:13.354936 24621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:13.355075 24570 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:06:13.355185 24570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "c4f9b8139a434eada46943ee89d6cc01"
format_stamp: "Formatted at 2026-08-06 15:05:52 on dist-test-slave-tc2s"
server_key: "cd4bc9e116c03dcb18a43f8029e62899"
server_key_iv: "9f899b0857a6cf3522c436f93677310b"
server_key_version: "encryptionkey@0"
I20260806 15:06:13.355499 24570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:13.384786 24570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:13.386787 24570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:13.386921 24570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:13.387115 24570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:06:13.387495 24630 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 15:06:13.388859 24570 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 15:06:13.388898 24570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 15:06:13.388929 24570 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 15:06:13.389523 24570 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 15:06:13.389555 24570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 15:06:13.389631 24630 tablet_bootstrap.cc:492] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01: Bootstrap starting.
I20260806 15:06:13.395983 24570 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:40075
I20260806 15:06:13.396075 24805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:40075 every 8 connection(s)
I20260806 15:06:13.396345 24570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:06:13.398157 24630 log.cc:824] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:13.400034 24630 tablet_bootstrap.cc:492] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 15:06:13.400278 24630 tablet_bootstrap.cc:492] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01: Bootstrap complete.
I20260806 15:06:13.401019 24630 ts_tablet_manager.cc:1397] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260806 15:06:13.402288 24630 raft_consensus.cc:348] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } } peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } }
I20260806 15:06:13.402594 24630 raft_consensus.cc:729] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4f9b8139a434eada46943ee89d6cc01, State: Initialized, Role: FOLLOWER
I20260806 15:06:13.402709 24630 consensus_queue.cc:260] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } } peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } }
I20260806 15:06:13.402958 24630 ts_tablet_manager.cc:1428] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
Aug 06 15:06:13 dist-test-slave-tc2s krb5kdc[23562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028773, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:06:13.403682 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 24570
I20260806 15:06:13.403995 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:37857
--local_ip_for_outbound_sockets=127.0.236.66
--tserver_master_addrs=127.0.236.126:39435
--webserver_port=32853
--webserver_interface=127.0.236.66
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:06:13.408689 24809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:13.396989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.65:50439)
0806 15:06:13.397145 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:13.397184 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:13.397651 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:13.398082 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:13.398086 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:13.398380 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:06:13.398564 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:13.398569 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.399649 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.399650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:13.400536 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:13.400539 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.401163 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.401164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:13.401308 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:13.401805 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:06:13.401814 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:06:13.403337 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:06:13.406765 (+ 3428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:13.406772 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:13.406774 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:13.407080 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:13.407360 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:13.407361 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:13.407362 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:13.407516 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:13.408057 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:06:13.408065 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:06:13.408216 (+ 151us) client_negotiation.cc:769] Sending connection context
0806 15:06:13.408330 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 15:06:13.408501 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260806 15:06:13.408725 24812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:13.397104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:50439 (local address 127.0.236.126:39435)
0806 15:06:13.397290 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:13.397298 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:13.397334 (+ 36us) server_negotiation.cc:407] Connection header received
0806 15:06:13.397844 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:13.397849 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:13.397922 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:13.398045 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:13.398676 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.399529 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.400614 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.401052 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.401157 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:13.403455 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:13.403459 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:13.403459 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:06:13.403473 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:13.406678 (+ 3205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:13.407167 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:13.407170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:13.407170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:13.407298 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:13.407579 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:13.407604 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:13.407604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:13.407971 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:13.408064 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:13.408402 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 15:06:13.408541 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":102,"threads_started":1}
I20260806 15:06:13.409404 24806 heartbeater.cc:347] Connected to a master server at 127.0.236.126:39435
I20260806 15:06:13.409494 24806 heartbeater.cc:464] Registering TS with master...
I20260806 15:06:13.409662 24806 heartbeater.cc:511] Master 127.0.236.126:39435 requested a full tablet report, sending...
I20260806 15:06:13.410418 24513 ts_manager.cc:194] Registered new tserver with Master: c4f9b8139a434eada46943ee89d6cc01 (127.0.236.65:40075)
I20260806 15:06:13.411562 24513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.65@KRBTEST.COM'} at 127.0.236.65:50439
I20260806 15:06:13.412568 24806 heartbeater.cc:503] Master 127.0.236.126:39435 was elected leader, sending a full tablet report...
I20260806 15:06:13.415361 24820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:13.411098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60364 (local address 127.0.236.65:40075)
0806 15:06:13.411327 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:13.411331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:13.412604 (+ 1273us) server_negotiation.cc:407] Connection header received
0806 15:06:13.412712 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:13.412714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:13.412728 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:13.412793 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:13.412938 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.413562 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.414556 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.415008 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.415069 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:13.415121 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:13.415262 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 15:06:13.415297 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":91,"threads_started":1}
I20260806 15:06:13.415364 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:13.411014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:40075 (local address 127.0.0.1:60364)
0806 15:06:13.412501 (+ 1487us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:13.412534 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:13.412604 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:13.412805 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:13.412808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:13.412811 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:13.412872 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:13.412874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.413634 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.413636 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:13.414481 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:13.414483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.415084 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.415085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:13.415171 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:13.415184 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 15:06:13.415243 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 15:06:13.415297 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1447,"thread_start_us":76,"threads_started":1}
I20260806 15:06:13.416038 24702 tablet_service.cc:1514] Processing DeleteTablet for tablet f26e2d9a473a44f780d8b70c84ca0573 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:05:53 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:60364
I20260806 15:06:13.416486 24821 tablet_replica.cc:330] stopping tablet replica
I20260806 15:06:13.416581 24821 raft_consensus.cc:2229] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:06:13.416641 24821 raft_consensus.cc:2258] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:06:13.416895 24821 ts_tablet_manager.cc:1905] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:06:13.418816 24821 ts_tablet_manager.cc:1918] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:06:13.418880 24821 log.cc:1196] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/wal/wals/f26e2d9a473a44f780d8b70c84ca0573
I20260806 15:06:13.419054 24821 ts_tablet_manager.cc:1939] T f26e2d9a473a44f780d8b70c84ca0573 P c4f9b8139a434eada46943ee89d6cc01: Deleting consensus metadata
I20260806 15:06:13.419452 24493 catalog_manager.cc:4841] TS c4f9b8139a434eada46943ee89d6cc01 (127.0.236.65:40075): tablet f26e2d9a473a44f780d8b70c84ca0573 (table test-table [id=b25f5472fa874cd9acc0c764a3c6830e]) successfully deleted
W20260806 15:06:13.486456 24817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:13.486694 24817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:13.486721 24817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:13.487933 24817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:13.487974 24817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:13.487999 24817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:06:13.489372 24817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:37857
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=32853
--tserver_master_addrs=127.0.236.126:39435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:13.489542 24817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:13.489714 24817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:13.494271 24829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:13.494282 24826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:13.494270 24827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:13.494491 24817 server_base.cc:1034] running on GCE node
I20260806 15:06:13.494675 24817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:13.494889 24817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:13.496016 24817 hybrid_clock.cc:648] HybridClock initialized: now 1786028773496008 us; error 27 us; skew 500 ppm
Aug 06 15:06:13 dist-test-slave-tc2s krb5kdc[23562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028773, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:06:13.498283 24817 init.cc:377] Logged in from keytab as kudu/127.0.236.66@KRBTEST.COM (short username kudu)
I20260806 15:06:13.498710 24817 webserver.cc:460] Webserver started at http://127.0.236.66:32853/ using document root <none> and password file <none>
I20260806 15:06:13.498894 24817 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:13.498950 24817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:13.500141 24817 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:06:13.500676 24841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:13.500834 24817 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:13.500914 24817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "9f2c60fb4db442c1b269ed12633570ee"
format_stamp: "Formatted at 2026-08-06 15:05:52 on dist-test-slave-tc2s"
server_key: "df6ebcfd7b4a59dd32d829454bad19ed"
server_key_iv: "98419b7e630623038e7b68c96bebf0bc"
server_key_version: "encryptionkey@0"
I20260806 15:06:13.501197 24817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:13.516412 24817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:13.518039 24817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:13.518157 24817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:13.518339 24817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:06:13.518610 24817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:06:13.518644 24817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:13.518669 24817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:06:13.518693 24817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:13.523836 24817 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:37857
I20260806 15:06:13.523874 25032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:37857 every 8 connection(s)
I20260806 15:06:13.524149 24817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
Aug 06 15:06:13 dist-test-slave-tc2s krb5kdc[23562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028773, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:06:13.530580 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 24817
I20260806 15:06:13.530827 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:37451
--local_ip_for_outbound_sockets=127.0.236.67
--tserver_master_addrs=127.0.236.126:39435
--webserver_port=43843
--webserver_interface=127.0.236.67
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:06:13.532344 24812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:13.524742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:55805 (local address 127.0.236.126:39435)
0806 15:06:13.524792 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:13.524794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:13.524964 (+ 170us) server_negotiation.cc:407] Connection header received
0806 15:06:13.525394 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:13.525395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:13.525412 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:13.525462 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:13.526324 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.526829 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.527797 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.528047 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.528103 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:13.529914 (+ 1811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:13.529916 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:13.529916 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:06:13.529923 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:13.531125 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:13.531555 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:13.531556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:13.531556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:13.531604 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:13.531822 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:13.531822 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:13.531823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:13.531918 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:13.531960 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:13.532218 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 15:06:13.532283 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 15:06:13.532562 25038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:13.524738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.66:55805)
0806 15:06:13.524866 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:13.524896 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:13.525302 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:13.525523 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:13.525526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:13.526069 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:06:13.526239 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:13.526242 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.526910 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.526911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:13.527698 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:13.527701 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.528119 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.528121 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:13.528226 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:13.528578 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:06:13.528585 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:06:13.529799 (+ 1214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:06:13.531196 (+ 1397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:13.531203 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:13.531205 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:13.531493 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:13.531657 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:13.531658 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:13.531658 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:13.531773 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:13.531965 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:06:13.531970 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:06:13.532108 (+ 138us) client_negotiation.cc:769] Sending connection context
0806 15:06:13.532208 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 15:06:13.532346 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":66,"threads_started":1}
I20260806 15:06:13.533021 25033 heartbeater.cc:347] Connected to a master server at 127.0.236.126:39435
I20260806 15:06:13.533102 25033 heartbeater.cc:464] Registering TS with master...
I20260806 15:06:13.533257 25033 heartbeater.cc:511] Master 127.0.236.126:39435 requested a full tablet report, sending...
I20260806 15:06:13.533607 24513 ts_manager.cc:194] Registered new tserver with Master: 9f2c60fb4db442c1b269ed12633570ee (127.0.236.66:37857)
I20260806 15:06:13.534013 24513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.66@KRBTEST.COM'} at 127.0.236.66:55805
W20260806 15:06:13.609937 25041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:13.610109 25041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:13.610139 25041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:13.611217 25041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:13.611255 25041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:13.611289 25041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:06:13.612690 25041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:37451
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=43843
--tserver_master_addrs=127.0.236.126:39435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:13.612905 25041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:13.613111 25041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 15:06:13.618075 25041 server_base.cc:1034] running on GCE node
W20260806 15:06:13.618060 25056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:13.618129 25052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:13.618053 25049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:13.618438 25041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:13.618661 25041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:13.619791 25041 hybrid_clock.cc:648] HybridClock initialized: now 1786028773619779 us; error 29 us; skew 500 ppm
Aug 06 15:06:13 dist-test-slave-tc2s krb5kdc[23562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028773, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:06:13.622126 25041 init.cc:377] Logged in from keytab as kudu/127.0.236.67@KRBTEST.COM (short username kudu)
I20260806 15:06:13.622480 25041 webserver.cc:460] Webserver started at http://127.0.236.67:43843/ using document root <none> and password file <none>
I20260806 15:06:13.622660 25041 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:13.622709 25041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:13.623881 25041 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:06:13.624356 25065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:13.624495 25041 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:13.624596 25041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "ea5947f4e19b4c6197aa84d31b44616c"
format_stamp: "Formatted at 2026-08-06 15:05:52 on dist-test-slave-tc2s"
server_key: "ac1c95289126702594a3afcfa5a2b565"
server_key_iv: "90a63a2e6ca10e22f08337c3c1f758cb"
server_key_version: "encryptionkey@0"
I20260806 15:06:13.624892 25041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:13.632732 25041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:13.634347 25041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:13.634450 25041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:13.634605 25041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:06:13.634825 25041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:06:13.634853 25041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:13.634871 25041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:06:13.634887 25041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:13.639943 25041 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:37451
I20260806 15:06:13.639998 25249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:37451 every 8 connection(s)
I20260806 15:06:13.640265 25041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
Aug 06 15:06:13 dist-test-slave-tc2s krb5kdc[23562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028773, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:06:13.646272 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 25041
I20260806 15:06:13.648698 24812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:13.640909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:57155 (local address 127.0.236.126:39435)
0806 15:06:13.640969 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:13.640971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:13.641132 (+ 161us) server_negotiation.cc:407] Connection header received
0806 15:06:13.641588 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:13.641590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:13.641602 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:13.641653 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:13.642559 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.643045 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.643983 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.644175 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.644230 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:13.646142 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:13.646144 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:13.646144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:06:13.646151 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:13.647490 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:13.647864 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:13.647865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:13.647865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:13.647926 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:13.648129 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:13.648129 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:13.648130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:13.648246 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:13.648304 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:13.648572 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 15:06:13.648635 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 15:06:13.648904 25253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:13.640882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.67:57155)
0806 15:06:13.641017 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:13.641055 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:13.641489 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:13.641714 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:13.641718 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:13.642280 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:06:13.642460 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:13.642465 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.643123 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.643124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:13.643894 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:13.643896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.644243 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.644244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:13.644367 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:13.644778 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:06:13.644787 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:06:13.646064 (+ 1277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:06:13.647567 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:13.647574 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:13.647576 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:13.647802 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:13.647991 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:13.647991 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:13.647992 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:13.648080 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:13.648300 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:06:13.648305 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:06:13.648449 (+ 144us) client_negotiation.cc:769] Sending connection context
0806 15:06:13.648555 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 15:06:13.648703 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":66,"threads_started":1}
I20260806 15:06:13.649386 25250 heartbeater.cc:347] Connected to a master server at 127.0.236.126:39435
I20260806 15:06:13.649478 25250 heartbeater.cc:464] Registering TS with master...
I20260806 15:06:13.649652 25250 heartbeater.cc:511] Master 127.0.236.126:39435 requested a full tablet report, sending...
I20260806 15:06:13.650046 24513 ts_manager.cc:194] Registered new tserver with Master: ea5947f4e19b4c6197aa84d31b44616c (127.0.236.67:37451)
I20260806 15:06:13.650384 24513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.67@KRBTEST.COM'} at 127.0.236.67:57155
I20260806 15:06:13.653321 25254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:13.646644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39094 (local address 127.0.236.126:39435)
0806 15:06:13.646742 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:13.646744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:13.646957 (+ 213us) server_negotiation.cc:407] Connection header received
0806 15:06:13.647012 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:13.647013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:13.647025 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:13.647092 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:13.647628 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.648204 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.649252 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.649545 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.649600 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:13.651309 (+ 1709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:13.651310 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:13.651311 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:06:13.651317 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:13.652324 (+ 1007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:13.652638 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:13.652639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:13.652640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:13.652696 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:13.652859 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:13.652860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:13.652861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:13.652966 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:13.653121 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:13.653157 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 15:06:13.653231 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":47,"threads_started":1}
I20260806 15:06:13.653936 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:06:13.656275 25254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:13.654234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39104 (local address 127.0.236.126:39435)
0806 15:06:13.654290 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:13.654292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:13.654369 (+ 77us) server_negotiation.cc:407] Connection header received
0806 15:06:13.654414 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:13.654415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:13.654452 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:13.654502 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:06:13.654564 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.654991 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.655681 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.655914 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.655957 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:13.656068 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:06:13.656146 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:06:13.656193 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:13.656205 (+ 12us) server_negotiation.cc:299] Negotiation successful
0806 15:06:13.656220 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 15:06:13.656527 24513 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:39104:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:06:13.658092 24513 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:06:13.660094 24702 tablet_service.cc:1467] Processing CreateTablet for tablet 593cafd5812143b7ae5cf9113c56aec3 (DEFAULT_TABLE table=test-table [id=ce0914f9f21343db8b156c9238b82fb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:06:13.660274 24702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 593cafd5812143b7ae5cf9113c56aec3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:13.661525 24630 tablet_bootstrap.cc:492] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01: Bootstrap starting.
I20260806 15:06:13.661839 24630 tablet_bootstrap.cc:654] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:13.662346 24630 tablet_bootstrap.cc:492] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01: No bootstrap required, opened a new log
I20260806 15:06:13.662389 24630 ts_tablet_manager.cc:1397] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:06:13.662496 24630 raft_consensus.cc:348] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } } peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } }
I20260806 15:06:13.662557 24630 raft_consensus.cc:374] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:13.662609 24630 raft_consensus.cc:729] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4f9b8139a434eada46943ee89d6cc01, State: Initialized, Role: FOLLOWER
I20260806 15:06:13.662653 24630 consensus_queue.cc:260] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } } peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } }
I20260806 15:06:13.662747 24630 ts_tablet_manager.cc:1428] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 15:06:13.664949 25264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:13.660022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54714 (local address 127.0.236.66:37857)
0806 15:06:13.660200 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:13.660204 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:13.661667 (+ 1463us) server_negotiation.cc:407] Connection header received
0806 15:06:13.661712 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:13.661715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:13.661732 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:13.661811 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:13.662007 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.663138 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.664020 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.664531 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.664674 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:13.664753 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:13.664830 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 15:06:13.664874 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":79,"threads_started":1}
I20260806 15:06:13.664915 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:13.659829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:37857 (local address 127.0.0.1:54714)
0806 15:06:13.661551 (+ 1722us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:13.661562 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:13.661622 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:13.661826 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:13.661827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:13.661830 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:13.661871 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:13.661872 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.663216 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.663217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:13.663934 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:13.663944 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.664677 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.664678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:13.664746 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:13.664750 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:06:13.664807 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 15:06:13.664846 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1708,"mutex_wait_us":2,"thread_start_us":85,"threads_started":1}
I20260806 15:06:13.665021 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:13.659788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:37451 (local address 127.0.0.1:54132)
0806 15:06:13.659832 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:13.659843 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:13.659880 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:13.661867 (+ 1987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:13.661868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:13.661870 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:13.661941 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:13.661943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.663442 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.663443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:13.664106 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:13.664107 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.664744 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.664745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:13.664807 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:13.664811 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:06:13.664864 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 15:06:13.664908 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":2}
I20260806 15:06:13.664995 25261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:13.659855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54132 (local address 127.0.236.67:37451)
0806 15:06:13.661722 (+ 1867us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:13.661726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:13.661734 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:06:13.661763 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:13.661765 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:13.661777 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:13.662014 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:13.662082 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.663365 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.664178 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.664665 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.664725 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:13.664777 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:13.664877 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 15:06:13.664922 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1838,"thread_start_us":80,"threads_started":1}
I20260806 15:06:13.665647 25144 tablet_service.cc:1467] Processing CreateTablet for tablet 593cafd5812143b7ae5cf9113c56aec3 (DEFAULT_TABLE table=test-table [id=ce0914f9f21343db8b156c9238b82fb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:06:13.665648 24925 tablet_service.cc:1467] Processing CreateTablet for tablet 593cafd5812143b7ae5cf9113c56aec3 (DEFAULT_TABLE table=test-table [id=ce0914f9f21343db8b156c9238b82fb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:06:13.666003 25144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 593cafd5812143b7ae5cf9113c56aec3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:13.666003 24925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 593cafd5812143b7ae5cf9113c56aec3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:13.669368 25266 tablet_bootstrap.cc:492] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee: Bootstrap starting.
I20260806 15:06:13.670027 25266 tablet_bootstrap.cc:654] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:13.670066 25267 tablet_bootstrap.cc:492] T 593cafd5812143b7ae5cf9113c56aec3 P ea5947f4e19b4c6197aa84d31b44616c: Bootstrap starting.
I20260806 15:06:13.670584 25266 log.cc:824] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:13.670715 25267 tablet_bootstrap.cc:654] T 593cafd5812143b7ae5cf9113c56aec3 P ea5947f4e19b4c6197aa84d31b44616c: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:13.671231 25267 log.cc:824] T 593cafd5812143b7ae5cf9113c56aec3 P ea5947f4e19b4c6197aa84d31b44616c: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:13.671356 25266 tablet_bootstrap.cc:492] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee: No bootstrap required, opened a new log
I20260806 15:06:13.671430 25266 ts_tablet_manager.cc:1397] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 15:06:13.671976 25267 tablet_bootstrap.cc:492] T 593cafd5812143b7ae5cf9113c56aec3 P ea5947f4e19b4c6197aa84d31b44616c: No bootstrap required, opened a new log
I20260806 15:06:13.672049 25267 ts_tablet_manager.cc:1397] T 593cafd5812143b7ae5cf9113c56aec3 P ea5947f4e19b4c6197aa84d31b44616c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:06:13.673259 25266 raft_consensus.cc:348] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } } peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } }
I20260806 15:06:13.673441 25266 raft_consensus.cc:374] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:13.673484 25266 raft_consensus.cc:729] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f2c60fb4db442c1b269ed12633570ee, State: Initialized, Role: FOLLOWER
I20260806 15:06:13.673604 25266 consensus_queue.cc:260] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } } peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } }
I20260806 15:06:13.673872 25033 heartbeater.cc:503] Master 127.0.236.126:39435 was elected leader, sending a full tablet report...
I20260806 15:06:13.673919 25266 ts_tablet_manager.cc:1428] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 15:06:13.673880 25267 raft_consensus.cc:348] T 593cafd5812143b7ae5cf9113c56aec3 P ea5947f4e19b4c6197aa84d31b44616c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } } peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } }
I20260806 15:06:13.674067 25267 raft_consensus.cc:374] T 593cafd5812143b7ae5cf9113c56aec3 P ea5947f4e19b4c6197aa84d31b44616c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:13.674098 25267 raft_consensus.cc:729] T 593cafd5812143b7ae5cf9113c56aec3 P ea5947f4e19b4c6197aa84d31b44616c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea5947f4e19b4c6197aa84d31b44616c, State: Initialized, Role: FOLLOWER
I20260806 15:06:13.674198 25267 consensus_queue.cc:260] T 593cafd5812143b7ae5cf9113c56aec3 P ea5947f4e19b4c6197aa84d31b44616c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } } peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } }
I20260806 15:06:13.674448 25267 ts_tablet_manager.cc:1428] T 593cafd5812143b7ae5cf9113c56aec3 P ea5947f4e19b4c6197aa84d31b44616c: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 15:06:13.674459 25250 heartbeater.cc:503] Master 127.0.236.126:39435 was elected leader, sending a full tablet report...
I20260806 15:06:13.700853 25270 raft_consensus.cc:482] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:06:13.700938 25270 raft_consensus.cc:504] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } } peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } }
I20260806 15:06:13.701220 25270 leader_election.cc:288] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea5947f4e19b4c6197aa84d31b44616c (127.0.236.67:37451), c4f9b8139a434eada46943ee89d6cc01 (127.0.236.65:40075)
I20260806 15:06:13.703783 25261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:13.701414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:58547 (local address 127.0.236.67:37451)
0806 15:06:13.701466 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:13.701468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:13.701497 (+ 29us) server_negotiation.cc:407] Connection header received
0806 15:06:13.701520 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:13.701521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:13.701529 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:13.701574 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:13.701700 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.702231 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.703108 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.703469 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.703530 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:13.703538 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:13.703692 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 15:06:13.703724 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 15:06:13.703805 25038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:13.701344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:37451 (local address 127.0.236.66:58547)
0806 15:06:13.701399 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:13.701412 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:13.701458 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:13.701585 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:13.701586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:13.701588 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:13.701646 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:13.701648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.702294 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.702295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:13.703037 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:13.703039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.703536 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.703537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:13.703634 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:13.703638 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:06:13.703686 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 15:06:13.703725 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 15:06:13.703924 25274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:13.701446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:40075 (local address 127.0.236.66:42607)
0806 15:06:13.701583 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:13.701593 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:13.701641 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:13.701759 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:13.701760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:13.701761 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:13.701827 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:13.701828 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.702415 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.702416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:13.703149 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:13.703151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.703692 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.703693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:13.703766 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:13.703770 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:06:13.703815 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 15:06:13.703851 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
I20260806 15:06:13.703929 24820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:13.701544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:42607 (local address 127.0.236.65:40075)
0806 15:06:13.701588 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:13.701590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:13.701633 (+ 43us) server_negotiation.cc:407] Connection header received
0806 15:06:13.701699 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:13.701699 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:13.701707 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:13.701750 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:13.701886 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.702348 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.703229 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.703620 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.703691 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:13.703700 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:13.703832 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 15:06:13.703872 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 15:06:13.704098 25184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "593cafd5812143b7ae5cf9113c56aec3" candidate_uuid: "9f2c60fb4db442c1b269ed12633570ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea5947f4e19b4c6197aa84d31b44616c" is_pre_election: true
I20260806 15:06:13.704221 25184 raft_consensus.cc:2454] T 593cafd5812143b7ae5cf9113c56aec3 P ea5947f4e19b4c6197aa84d31b44616c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f2c60fb4db442c1b269ed12633570ee in term 0.
I20260806 15:06:13.704432 24847 leader_election.cc:302] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f2c60fb4db442c1b269ed12633570ee, ea5947f4e19b4c6197aa84d31b44616c; no voters:
I20260806 15:06:13.704547 25270 raft_consensus.cc:2781] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:06:13.704535 24734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "593cafd5812143b7ae5cf9113c56aec3" candidate_uuid: "9f2c60fb4db442c1b269ed12633570ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4f9b8139a434eada46943ee89d6cc01" is_pre_election: true
I20260806 15:06:13.704622 25270 raft_consensus.cc:482] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:06:13.704648 25270 raft_consensus.cc:3037] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:13.704668 24734 raft_consensus.cc:2454] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f2c60fb4db442c1b269ed12633570ee in term 0.
I20260806 15:06:13.705399 25270 raft_consensus.cc:504] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } } peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } }
I20260806 15:06:13.705504 25270 leader_election.cc:288] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee [CANDIDATE]: Term 1 election: Requested vote from peers ea5947f4e19b4c6197aa84d31b44616c (127.0.236.67:37451), c4f9b8139a434eada46943ee89d6cc01 (127.0.236.65:40075)
I20260806 15:06:13.705623 25184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "593cafd5812143b7ae5cf9113c56aec3" candidate_uuid: "9f2c60fb4db442c1b269ed12633570ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea5947f4e19b4c6197aa84d31b44616c"
I20260806 15:06:13.705647 24734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "593cafd5812143b7ae5cf9113c56aec3" candidate_uuid: "9f2c60fb4db442c1b269ed12633570ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4f9b8139a434eada46943ee89d6cc01"
I20260806 15:06:13.705698 25184 raft_consensus.cc:3037] T 593cafd5812143b7ae5cf9113c56aec3 P ea5947f4e19b4c6197aa84d31b44616c [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:13.705708 24734 raft_consensus.cc:3037] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:13.706344 25184 raft_consensus.cc:2454] T 593cafd5812143b7ae5cf9113c56aec3 P ea5947f4e19b4c6197aa84d31b44616c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f2c60fb4db442c1b269ed12633570ee in term 1.
I20260806 15:06:13.706497 24847 leader_election.cc:302] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f2c60fb4db442c1b269ed12633570ee, ea5947f4e19b4c6197aa84d31b44616c; no voters:
I20260806 15:06:13.706580 25270 raft_consensus.cc:2781] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:06:13.706697 25270 raft_consensus.cc:686] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee [term 1 LEADER]: Becoming Leader. State: Replica: 9f2c60fb4db442c1b269ed12633570ee, State: Running, Role: LEADER
I20260806 15:06:13.706780 25270 consensus_queue.cc:237] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } } peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } }
I20260806 15:06:13.707032 24734 raft_consensus.cc:2454] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f2c60fb4db442c1b269ed12633570ee in term 1.
I20260806 15:06:13.707508 24513 catalog_manager.cc:5495] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f2c60fb4db442c1b269ed12633570ee (127.0.236.66). New cstate: current_term: 1 leader_uuid: "9f2c60fb4db442c1b269ed12633570ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } health_report { overall_health: UNKNOWN } } }
I20260806 15:06:13.721043 25264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:13.719155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54720 (local address 127.0.236.66:37857)
0806 15:06:13.719190 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:13.719192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:13.719238 (+ 46us) server_negotiation.cc:407] Connection header received
0806 15:06:13.719294 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:13.719295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:13.719308 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:13.719360 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:06:13.719438 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.719855 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.720496 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:13.720662 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:13.720704 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:13.720814 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:06:13.720894 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:06:13.720953 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:13.720964 (+ 11us) server_negotiation.cc:299] Negotiation successful
0806 15:06:13.720979 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:06:13.722556 24734 raft_consensus.cc:1261] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Refusing update from remote peer 9f2c60fb4db442c1b269ed12633570ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:06:13.722556 25184 raft_consensus.cc:1261] T 593cafd5812143b7ae5cf9113c56aec3 P ea5947f4e19b4c6197aa84d31b44616c [term 1 FOLLOWER]: Refusing update from remote peer 9f2c60fb4db442c1b269ed12633570ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:06:13.722853 25270 consensus_queue.cc:1035] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:06:13.722980 25275 consensus_queue.cc:1035] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:06:13.725518 25280 mvcc.cc:205] Tried to move back new op lower bound from 7315573857165361152 to 7315573857103560704. Current Snapshot: MvccSnapshot[applied={T|T < 7315573857165361152}]
I20260806 15:06:13.725572 25281 mvcc.cc:205] Tried to move back new op lower bound from 7315573857165361152 to 7315573857103560704. Current Snapshot: MvccSnapshot[applied={T|T < 7315573857165361152}]
I20260806 15:06:13.726974 24513 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:39104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:06:13.727053 24513 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:39104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:06:13.727728 24513 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24: Sending DeleteTablet for 3 replicas of tablet 593cafd5812143b7ae5cf9113c56aec3
I20260806 15:06:13.728009 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 24570
I20260806 15:06:13.728034 24702 tablet_service.cc:1514] Processing DeleteTablet for tablet 593cafd5812143b7ae5cf9113c56aec3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:06:13 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:60364
I20260806 15:06:13.728044 25144 tablet_service.cc:1514] Processing DeleteTablet for tablet 593cafd5812143b7ae5cf9113c56aec3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:06:13 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:54132
I20260806 15:06:13.728117 24821 tablet_replica.cc:330] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01: stopping tablet replica
I20260806 15:06:13.728166 24821 raft_consensus.cc:2229] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:06:13.728210 24821 raft_consensus.cc:2258] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:06:13.736384 24925 tablet_service.cc:1514] Processing DeleteTablet for tablet 593cafd5812143b7ae5cf9113c56aec3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:06:13 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:54714
I20260806 15:06:13.736603 25296 tablet_replica.cc:330] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee: stopping tablet replica
I20260806 15:06:13.736707 25296 raft_consensus.cc:2229] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:06:13.736822 25296 raft_consensus.cc:2258] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:06:13.737028 25294 tablet_replica.cc:330] T 593cafd5812143b7ae5cf9113c56aec3 P ea5947f4e19b4c6197aa84d31b44616c: stopping tablet replica
I20260806 15:06:13.737129 25294 raft_consensus.cc:2229] T 593cafd5812143b7ae5cf9113c56aec3 P ea5947f4e19b4c6197aa84d31b44616c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:06:13.737234 25294 raft_consensus.cc:2258] T 593cafd5812143b7ae5cf9113c56aec3 P ea5947f4e19b4c6197aa84d31b44616c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:06:13.737227 25296 ts_tablet_manager.cc:1905] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:06:13.738900 25296 ts_tablet_manager.cc:1918] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:06:13.738986 25296 log.cc:1196] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/wal/wals/593cafd5812143b7ae5cf9113c56aec3
I20260806 15:06:13.739255 25296 ts_tablet_manager.cc:1939] T 593cafd5812143b7ae5cf9113c56aec3 P 9f2c60fb4db442c1b269ed12633570ee: Deleting consensus metadata
I20260806 15:06:13.739631 24489 catalog_manager.cc:4841] TS 9f2c60fb4db442c1b269ed12633570ee (127.0.236.66:37857): tablet 593cafd5812143b7ae5cf9113c56aec3 (table test-table [id=ce0914f9f21343db8b156c9238b82fb6]) successfully deleted
W20260806 15:06:13.742197 24493 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.236.65:40075 (error 108)
I20260806 15:06:13.742600 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 24817
I20260806 15:06:13.742624 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:13.742504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:40075 (local address 127.0.0.1:60376)
0806 15:06:13.742546 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:13.742591 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.65:40075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260806 15:06:13.745002 25294 ts_tablet_manager.cc:1905] T 593cafd5812143b7ae5cf9113c56aec3 P ea5947f4e19b4c6197aa84d31b44616c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:06:13.746981 25294 ts_tablet_manager.cc:1918] T 593cafd5812143b7ae5cf9113c56aec3 P ea5947f4e19b4c6197aa84d31b44616c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:06:13.747035 25294 log.cc:1196] T 593cafd5812143b7ae5cf9113c56aec3 P ea5947f4e19b4c6197aa84d31b44616c: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/593cafd5812143b7ae5cf9113c56aec3
I20260806 15:06:13.747242 25294 ts_tablet_manager.cc:1939] T 593cafd5812143b7ae5cf9113c56aec3 P ea5947f4e19b4c6197aa84d31b44616c: Deleting consensus metadata
W20260806 15:06:13.748343 24493 catalog_manager.cc:4584] TS c4f9b8139a434eada46943ee89d6cc01 (127.0.236.65:40075): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 593cafd5812143b7ae5cf9113c56aec3: Network error: Client connection negotiation failed: client connection to 127.0.236.65:40075: connect: Connection refused (error 111)
I20260806 15:06:13.748725 24493 catalog_manager.cc:4841] TS ea5947f4e19b4c6197aa84d31b44616c (127.0.236.67:37451): tablet 593cafd5812143b7ae5cf9113c56aec3 (table test-table [id=ce0914f9f21343db8b156c9238b82fb6]) successfully deleted
I20260806 15:06:13.758241 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 25041
I20260806 15:06:13.774609 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 24458
I20260806 15:06:33.781222 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:39435
--webserver_interface=127.0.236.126
--webserver_port=42213
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:39435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:06:33.861022 25298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:33.861202 25298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:33.861231 25298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:33.862352 25298 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 15:06:33.862388 25298 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 15:06:33.862406 25298 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 15:06:33.862421 25298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:06:33.862437 25298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:33.862458 25298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:33.862483 25298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:06:33.862500 25298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:06:33.863931 25298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:39435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:39435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=42213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:33.864156 25298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:33.864354 25298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:33.868902 25305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:33.868903 25308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:33.868974 25298 server_base.cc:1034] running on GCE node
W20260806 15:06:33.868945 25306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:33.869321 25298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:33.869620 25298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:33.870754 25298 hybrid_clock.cc:648] HybridClock initialized: now 1786028793870740 us; error 28 us; skew 500 ppm
Aug 06 15:06:33 dist-test-slave-tc2s krb5kdc[23562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028793, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:06:33.877669 25298 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:06:33.878096 25298 webserver.cc:460] Webserver started at http://127.0.236.126:42213/ using document root <none> and password file <none>
I20260806 15:06:33.878283 25298 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:33.878321 25298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:33.879465 25298 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:06:33.880033 25315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:33.880188 25298 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:06:33.880267 25298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "b87064a86f4b49fe85d745118a59df24"
format_stamp: "Formatted at 2026-08-06 15:05:52 on dist-test-slave-tc2s"
server_key: "b1875e549dcda48f5344a81eed1dbc1e"
server_key_iv: "38cad7a2209da84bd4af2e6f2a352fd4"
server_key_version: "encryptionkey@0"
I20260806 15:06:33.880553 25298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:33.898494 25298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:33.900413 25298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:33.900516 25298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:33.904206 25298 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:39435
I20260806 15:06:33.904237 25397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:39435 every 8 connection(s)
I20260806 15:06:33.904564 25298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:06:33.905988 25398 sys_catalog.cc:263] Verifying existing consensus state
I20260806 15:06:33.906518 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 25298
I20260806 15:06:33.906690 25398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24: Bootstrap starting.
I20260806 15:06:33.906802 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:40075
--local_ip_for_outbound_sockets=127.0.236.65
--tserver_master_addrs=127.0.236.126:39435
--webserver_port=45135
--webserver_interface=127.0.236.65
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:06:33.915019 25398 log.cc:824] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:33.916972 25398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 15:06:33.917205 25398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24: Bootstrap complete.
I20260806 15:06:33.919176 25398 raft_consensus.cc:348] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87064a86f4b49fe85d745118a59df24" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39435 } }
I20260806 15:06:33.919386 25398 raft_consensus.cc:729] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b87064a86f4b49fe85d745118a59df24, State: Initialized, Role: FOLLOWER
I20260806 15:06:33.919526 25398 consensus_queue.cc:260] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87064a86f4b49fe85d745118a59df24" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39435 } }
I20260806 15:06:33.919579 25398 raft_consensus.cc:388] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:06:33.919615 25398 raft_consensus.cc:482] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:06:33.919659 25398 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [term 2 FOLLOWER]: Advancing to term 3
I20260806 15:06:33.920558 25398 raft_consensus.cc:504] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87064a86f4b49fe85d745118a59df24" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39435 } }
I20260806 15:06:33.920642 25398 leader_election.cc:302] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b87064a86f4b49fe85d745118a59df24; no voters:
I20260806 15:06:33.920807 25398 leader_election.cc:288] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 15:06:33.920863 25402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 15:06:33.921020 25402 raft_consensus.cc:686] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [term 3 LEADER]: Becoming Leader. State: Replica: b87064a86f4b49fe85d745118a59df24, State: Running, Role: LEADER
I20260806 15:06:33.921058 25398 sys_catalog.cc:564] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:06:33.921118 25402 consensus_queue.cc:237] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87064a86f4b49fe85d745118a59df24" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39435 } }
I20260806 15:06:33.921401 25403 sys_catalog.cc:455] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b87064a86f4b49fe85d745118a59df24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87064a86f4b49fe85d745118a59df24" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39435 } } }
I20260806 15:06:33.921414 25404 sys_catalog.cc:455] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b87064a86f4b49fe85d745118a59df24. Latest consensus state: current_term: 3 leader_uuid: "b87064a86f4b49fe85d745118a59df24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87064a86f4b49fe85d745118a59df24" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 39435 } } }
I20260806 15:06:33.921494 25403 sys_catalog.cc:458] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [sys.catalog]: This master's current role is: LEADER
I20260806 15:06:33.921496 25404 sys_catalog.cc:458] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24 [sys.catalog]: This master's current role is: LEADER
I20260806 15:06:33.921777 25415 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:06:33.922713 25415 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:06:33.922816 25415 catalog_manager.cc:1259] Loaded cluster ID: 9f18e9a24e384ccabec49e161db8d837
I20260806 15:06:33.922850 25415 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:06:33.923686 25415 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:06:33.923820 25415 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24: Loaded TSK: 0
I20260806 15:06:33.923851 25415 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b87064a86f4b49fe85d745118a59df24: Loaded TSK: 1
F20260806 15:06:33.924005 25415 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa0e78459ad google::LogMessage::Fail() at ??:0
@ 0x7fa0e784888a google::LogMessage::SendToLog() at ??:0
@ 0x7fa0e78454de google::LogMessage::Flush() at ??:0
@ 0x7fa0e7846a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa0e9fb2104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fa0e9fafd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fa0ef79f1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fa0ef79f48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fa0ef79f594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fa0ef765c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fa0ef76fad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa0e97130d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa0e9708954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa0ea56c6db start_thread at ??:0
@ 0x7fa0e634361f clone at ??:0
W20260806 15:06:34.001498 25400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:34.001686 25400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:34.001718 25400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:34.002851 25400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:34.002890 25400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:34.002918 25400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:06:34.004340 25400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:40075
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=45135
--tserver_master_addrs=127.0.236.126:39435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:34.004551 25400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:34.004745 25400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:34.009286 25461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:34.009424 25465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:34.009287 25462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:34.009649 25400 server_base.cc:1034] running on GCE node
I20260806 15:06:34.009816 25400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:34.010051 25400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:34.011185 25400 hybrid_clock.cc:648] HybridClock initialized: now 1786028794011172 us; error 29 us; skew 500 ppm
Aug 06 15:06:34 dist-test-slave-tc2s krb5kdc[23562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028794, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:06:34.013437 25400 init.cc:377] Logged in from keytab as kudu/127.0.236.65@KRBTEST.COM (short username kudu)
I20260806 15:06:34.013818 25400 webserver.cc:460] Webserver started at http://127.0.236.65:45135/ using document root <none> and password file <none>
I20260806 15:06:34.014031 25400 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:34.014079 25400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:34.015316 25400 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:06:34.015961 25479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:34.016105 25400 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:06:34.016184 25400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "c4f9b8139a434eada46943ee89d6cc01"
format_stamp: "Formatted at 2026-08-06 15:05:52 on dist-test-slave-tc2s"
server_key: "cd4bc9e116c03dcb18a43f8029e62899"
server_key_iv: "9f899b0857a6cf3522c436f93677310b"
server_key_version: "encryptionkey@0"
I20260806 15:06:34.016467 25400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:34.028095 25400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:34.030045 25400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:34.030169 25400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:34.030355 25400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:06:34.030740 25491 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 15:06:34.032503 25400 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 15:06:34.032543 25400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 15:06:34.032572 25400 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 15:06:34.033136 25400 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 15:06:34.033169 25400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 15:06:34.033255 25491 tablet_bootstrap.cc:492] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01: Bootstrap starting.
I20260806 15:06:34.039601 25400 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:40075
I20260806 15:06:34.039722 25661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:40075 every 8 connection(s)
I20260806 15:06:34.039974 25400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:06:34.042218 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 25400
I20260806 15:06:34.042467 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:37857
--local_ip_for_outbound_sockets=127.0.236.66
--tserver_master_addrs=127.0.236.126:39435
--webserver_port=32853
--webserver_interface=127.0.236.66
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:06:34.043677 25491 log.cc:824] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:34.045948 25491 tablet_bootstrap.cc:492] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 15:06:34.046190 25491 tablet_bootstrap.cc:492] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01: Bootstrap complete.
I20260806 15:06:34.046919 25491 ts_tablet_manager.cc:1397] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.011s
I20260806 15:06:34.047850 25491 raft_consensus.cc:348] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } } peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } }
I20260806 15:06:34.048063 25491 raft_consensus.cc:729] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4f9b8139a434eada46943ee89d6cc01, State: Initialized, Role: FOLLOWER
I20260806 15:06:34.048194 25491 consensus_queue.cc:260] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } } peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } }
I20260806 15:06:34.048450 25491 ts_tablet_manager.cc:1428] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01: Time spent starting tablet: real 0.001s user 0.001s sys 0.002s
W20260806 15:06:34.131712 25664 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:06:34.040572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.65:59797)
0806 15:06:34.040724 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:34.040762 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:34.041165 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:34.131518 (+ 90353us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
W20260806 15:06:34.131954 25662 heartbeater.cc:644] Failed to heartbeat to 127.0.236.126:39435 (0 consecutive failures): Network error: Failed to ping master at 127.0.236.126:39435: Client connection negotiation failed: client connection to 127.0.236.126:39435: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 15:06:34.132323 25664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:34.132213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.65:48501)
0806 15:06:34.132263 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:34.132300 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 15:06:34.132603 25664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:34.132530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.65:58921)
0806 15:06:34.132566 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:34.132587 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 15:06:34.132702 25662 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 15:06:34.149859 25666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:34.150079 25666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:34.150112 25666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:34.151212 25666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:34.151248 25666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:34.151283 25666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:06:34.152690 25666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:37857
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=32853
--tserver_master_addrs=127.0.236.126:39435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:34.152920 25666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:34.153107 25666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:34.157645 25674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:34.157634 25675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:34.157665 25666 server_base.cc:1034] running on GCE node
W20260806 15:06:34.157646 25677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:34.158033 25666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:34.158265 25666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:34.159408 25666 hybrid_clock.cc:648] HybridClock initialized: now 1786028794159391 us; error 27 us; skew 500 ppm
Aug 06 15:06:34 dist-test-slave-tc2s krb5kdc[23562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028794, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:06:34.161586 25666 init.cc:377] Logged in from keytab as kudu/127.0.236.66@KRBTEST.COM (short username kudu)
I20260806 15:06:34.162035 25666 webserver.cc:460] Webserver started at http://127.0.236.66:32853/ using document root <none> and password file <none>
I20260806 15:06:34.162237 25666 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:34.162285 25666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:34.163455 25666 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:06:34.163950 25684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:34.164091 25666 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:34.164170 25666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "9f2c60fb4db442c1b269ed12633570ee"
format_stamp: "Formatted at 2026-08-06 15:05:52 on dist-test-slave-tc2s"
server_key: "df6ebcfd7b4a59dd32d829454bad19ed"
server_key_iv: "98419b7e630623038e7b68c96bebf0bc"
server_key_version: "encryptionkey@0"
I20260806 15:06:34.164458 25666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:34.182641 25666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:34.184198 25666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:34.184319 25666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:34.184504 25666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:06:34.184762 25666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:06:34.184800 25666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:34.184827 25666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:06:34.184850 25666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:34.190340 25666 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:37857
I20260806 15:06:34.190376 25857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:37857 every 8 connection(s)
I20260806 15:06:34.190675 25666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
I20260806 15:06:34.191572 25867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:34.191235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.66:60997)
0806 15:06:34.191382 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:34.191443 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":74,"threads_started":1}
W20260806 15:06:34.191887 25863 heartbeater.cc:644] Failed to heartbeat to 127.0.236.126:39435 (0 consecutive failures): Network error: Failed to ping master at 127.0.236.126:39435: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
I20260806 15:06:34.192121 25867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:34.192043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.66:53463)
0806 15:06:34.192081 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:34.192104 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 15:06:34.192309 25867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:34.192265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.66:44885)
0806 15:06:34.192283 (+ 18us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:34.192298 (+ 15us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10}
W20260806 15:06:34.192391 25863 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 15:06:34.198709 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 25666
I20260806 15:06:34.198946 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:37451
--local_ip_for_outbound_sockets=127.0.236.67
--tserver_master_addrs=127.0.236.126:39435
--webserver_port=43843
--webserver_interface=127.0.236.67
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:06:34.279767 25868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:34.279948 25868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:34.279971 25868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:34.281073 25868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:34.281112 25868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:34.281141 25868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:06:34.282557 25868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:37451
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=43843
--tserver_master_addrs=127.0.236.126:39435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:34.282747 25868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:34.282939 25868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 15:06:34.287529 25868 server_base.cc:1034] running on GCE node
W20260806 15:06:34.287508 25875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:34.287513 25878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:34.287524 25876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:34.287858 25868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:34.288069 25868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:34.289203 25868 hybrid_clock.cc:648] HybridClock initialized: now 1786028794289184 us; error 30 us; skew 500 ppm
Aug 06 15:06:34 dist-test-slave-tc2s krb5kdc[23562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028794, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:06:34.291603 25868 init.cc:377] Logged in from keytab as kudu/127.0.236.67@KRBTEST.COM (short username kudu)
I20260806 15:06:34.291994 25868 webserver.cc:460] Webserver started at http://127.0.236.67:43843/ using document root <none> and password file <none>
I20260806 15:06:34.292174 25868 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:34.292220 25868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:34.293361 25868 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:06:34.293936 25891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:34.294085 25868 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:34.294163 25868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "ea5947f4e19b4c6197aa84d31b44616c"
format_stamp: "Formatted at 2026-08-06 15:05:52 on dist-test-slave-tc2s"
server_key: "ac1c95289126702594a3afcfa5a2b565"
server_key_iv: "90a63a2e6ca10e22f08337c3c1f758cb"
server_key_version: "encryptionkey@0"
I20260806 15:06:34.294443 25868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:34.308974 25868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:34.310573 25868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:34.310691 25868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:34.310873 25868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:06:34.311133 25868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:06:34.311164 25868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:34.311190 25868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:06:34.311208 25868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:34.316349 25868 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:37451
I20260806 15:06:34.316404 26055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:37451 every 8 connection(s)
I20260806 15:06:34.316684 25868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
I20260806 15:06:34.317546 26059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:34.317226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.67:59225)
0806 15:06:34.317354 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:34.317415 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":65,"threads_started":1}
W20260806 15:06:34.317867 26057 heartbeater.cc:644] Failed to heartbeat to 127.0.236.126:39435 (0 consecutive failures): Network error: Failed to ping master at 127.0.236.126:39435: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
I20260806 15:06:34.318130 26059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:34.318049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.67:33733)
0806 15:06:34.318086 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:34.318112 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260806 15:06:34.318325 26059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:34.318267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.67:37227)
0806 15:06:34.318295 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:34.318313 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260806 15:06:34.318401 26057 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 15:06:34.324177 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 25868
I20260806 15:06:35.133184 26067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:35.132984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.65:50557)
0806 15:06:35.133095 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:35.133125 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":59,"threads_started":1}
I20260806 15:06:35.192792 26069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:35.192626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.66:59165)
0806 15:06:35.192724 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:35.192751 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":51,"threads_started":1}
I20260806 15:06:35.237975 26070 raft_consensus.cc:482] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:06:35.238083 26070 raft_consensus.cc:504] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } } peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } }
I20260806 15:06:35.238349 26070 leader_election.cc:288] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ea5947f4e19b4c6197aa84d31b44616c (127.0.236.67:37451), 9f2c60fb4db442c1b269ed12633570ee (127.0.236.66:37857)
Aug 06 15:06:35 dist-test-slave-tc2s krb5kdc[23562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028794, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for kudu/127.0.236.67@KRBTEST.COM
Aug 06 15:06:35 dist-test-slave-tc2s krb5kdc[23562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028794, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for kudu/127.0.236.66@KRBTEST.COM
I20260806 15:06:35.247810 26072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:35.238580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:56849 (local address 127.0.236.67:37451)
0806 15:06:35.238736 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:35.238750 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:35.238774 (+ 24us) server_negotiation.cc:407] Connection header received
0806 15:06:35.239231 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:35.239233 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:35.239282 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:35.239399 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:35.240151 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:35.241015 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:35.242138 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:35.242576 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:35.242672 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:35.244764 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:35.244768 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:35.244768 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:06:35.244778 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:35.246352 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:35.246719 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:35.246722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:35.246722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:35.246809 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:35.247032 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:35.247033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:35.247033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:35.247258 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:35.247328 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:35.247601 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 15:06:35.247719 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":58,"threads_started":1}
I20260806 15:06:35.247810 26067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:35.238481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:37451 (local address 127.0.236.65:56849)
0806 15:06:35.238719 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:35.238730 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:35.238787 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:35.239436 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:35.239439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:35.239838 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:06:35.240037 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:35.240043 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:35.241119 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:35.241120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:35.241976 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:35.241978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:35.242649 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:35.242651 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:35.242722 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:35.243213 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:06:35.243221 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:06:35.244666 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:06:35.246425 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:35.246433 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:35.246435 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:35.246643 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:35.246868 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:35.246869 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:35.246869 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:35.246976 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:35.247321 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:06:35.247328 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:06:35.247452 (+ 124us) client_negotiation.cc:769] Sending connection context
0806 15:06:35.247536 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 15:06:35.247701 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":49}
I20260806 15:06:35.247896 26071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:35.238496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:37857 (local address 127.0.236.65:58781)
0806 15:06:35.238613 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:35.238621 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:35.238666 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:35.239444 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:35.239446 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:35.239837 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:06:35.240037 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:35.240042 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:35.241099 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:35.241099 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:35.241977 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:35.241980 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:35.242608 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:35.242609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:35.242716 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:35.243256 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:06:35.243260 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:06:35.245037 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:06:35.246766 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:35.246766 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:35.246767 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:35.246931 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:35.247140 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:35.247141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:35.247141 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:35.247200 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:35.247530 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:06:35.247531 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:06:35.247574 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 15:06:35.247642 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 15:06:35.247766 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":97,"thread_start_us":55,"threads_started":1}
I20260806 15:06:35.247938 26073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:35.238601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:58781 (local address 127.0.236.66:37857)
0806 15:06:35.238754 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:35.238763 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:35.238789 (+ 26us) server_negotiation.cc:407] Connection header received
0806 15:06:35.239206 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:35.239208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:35.239246 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:35.239368 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:35.240142 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:35.241007 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:35.242060 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:35.242527 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:35.242624 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:35.245124 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:35.245128 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:35.245128 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:06:35.245141 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:35.246688 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:35.246997 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:35.247000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:35.247000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:35.247084 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:35.247249 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:35.247250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:35.247250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:35.247468 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:35.247537 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:35.247706 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 15:06:35.247829 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":59,"threads_started":1}
W20260806 15:06:35.248728 25481 leader_election.cc:341] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9f2c60fb4db442c1b269ed12633570ee (127.0.236.66:37857): Not found: Tablet not found: 593cafd5812143b7ae5cf9113c56aec3
W20260806 15:06:35.248824 25485 leader_election.cc:341] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ea5947f4e19b4c6197aa84d31b44616c (127.0.236.67:37451): Not found: Tablet not found: 593cafd5812143b7ae5cf9113c56aec3
I20260806 15:06:35.248872 25485 leader_election.cc:302] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4f9b8139a434eada46943ee89d6cc01; no voters: 9f2c60fb4db442c1b269ed12633570ee, ea5947f4e19b4c6197aa84d31b44616c
I20260806 15:06:35.248963 26070 raft_consensus.cc:2726] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 15:06:35.318835 26074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:35.318661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.67:36137)
0806 15:06:35.318760 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:35.318800 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":55,"threads_started":1}
I20260806 15:06:36.133729 26075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:36.133536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.65:59209)
0806 15:06:36.133651 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:36.133684 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":64,"threads_started":1}
I20260806 15:06:36.193369 26076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:36.193197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.66:54949)
0806 15:06:36.193303 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:36.193333 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":56,"threads_started":1}
I20260806 15:06:36.319360 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:36.319182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.67:59721)
0806 15:06:36.319285 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:36.319320 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":53,"threads_started":1}
I20260806 15:06:37.134266 26078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:37.134065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.65:44409)
0806 15:06:37.134189 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:37.134219 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":72,"threads_started":1}
I20260806 15:06:37.193883 26080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:37.193717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.66:40873)
0806 15:06:37.193819 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:37.193850 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":58,"threads_started":1}
I20260806 15:06:37.319891 26081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:37.319689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.67:40359)
0806 15:06:37.319819 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:37.319854 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":62,"threads_started":1}
I20260806 15:06:37.490491 26082 raft_consensus.cc:482] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:06:37.490576 26082 raft_consensus.cc:504] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } } peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } }
I20260806 15:06:37.490731 26082 leader_election.cc:288] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ea5947f4e19b4c6197aa84d31b44616c (127.0.236.67:37451), 9f2c60fb4db442c1b269ed12633570ee (127.0.236.66:37857)
W20260806 15:06:37.491118 25481 leader_election.cc:341] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9f2c60fb4db442c1b269ed12633570ee (127.0.236.66:37857): Not found: Tablet not found: 593cafd5812143b7ae5cf9113c56aec3
W20260806 15:06:37.491204 25485 leader_election.cc:341] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ea5947f4e19b4c6197aa84d31b44616c (127.0.236.67:37451): Not found: Tablet not found: 593cafd5812143b7ae5cf9113c56aec3
I20260806 15:06:37.491240 25485 leader_election.cc:302] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4f9b8139a434eada46943ee89d6cc01; no voters: 9f2c60fb4db442c1b269ed12633570ee, ea5947f4e19b4c6197aa84d31b44616c
I20260806 15:06:37.491317 26082 raft_consensus.cc:2726] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 15:06:38.134829 26083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:38.134625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.65:36023)
0806 15:06:38.134747 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:38.134781 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":67,"threads_started":1}
I20260806 15:06:38.194416 26084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:38.194239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.66:36329)
0806 15:06:38.194345 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:38.194377 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":57,"threads_started":1}
I20260806 15:06:38.320473 26086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:38.320283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.67:51993)
0806 15:06:38.320404 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:38.320436 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":71,"threads_started":1}
I20260806 15:06:39.135434 26087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:39.135233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.65:35791)
0806 15:06:39.135356 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:39.135386 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":74,"threads_started":1}
I20260806 15:06:39.194962 26090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:39.194779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.66:51077)
0806 15:06:39.194893 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:39.194925 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":60,"threads_started":1}
I20260806 15:06:39.321031 26092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:39.320821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.67:50473)
0806 15:06:39.320942 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:39.320981 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":68,"threads_started":1}
I20260806 15:06:39.789399 26093 raft_consensus.cc:482] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:06:39.789494 26093 raft_consensus.cc:504] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } } peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } }
I20260806 15:06:39.789655 26093 leader_election.cc:288] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ea5947f4e19b4c6197aa84d31b44616c (127.0.236.67:37451), 9f2c60fb4db442c1b269ed12633570ee (127.0.236.66:37857)
W20260806 15:06:39.790019 25481 leader_election.cc:341] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9f2c60fb4db442c1b269ed12633570ee (127.0.236.66:37857): Not found: Tablet not found: 593cafd5812143b7ae5cf9113c56aec3
W20260806 15:06:39.790095 25485 leader_election.cc:341] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ea5947f4e19b4c6197aa84d31b44616c (127.0.236.67:37451): Not found: Tablet not found: 593cafd5812143b7ae5cf9113c56aec3
I20260806 15:06:39.790163 25485 leader_election.cc:302] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4f9b8139a434eada46943ee89d6cc01; no voters: 9f2c60fb4db442c1b269ed12633570ee, ea5947f4e19b4c6197aa84d31b44616c
I20260806 15:06:39.790225 26093 raft_consensus.cc:2726] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 15:06:40.136013 26094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:40.135815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.65:58425)
0806 15:06:40.135928 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:40.135966 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":63,"threads_started":1}
I20260806 15:06:40.195485 26095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:40.195293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.66:55489)
0806 15:06:40.195403 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:40.195448 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":56,"threads_started":1}
I20260806 15:06:40.321563 26096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:40.321384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.67:54775)
0806 15:06:40.321491 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:40.321524 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":60,"threads_started":1}
I20260806 15:06:41.136595 26098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:41.136392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.65:55279)
0806 15:06:41.136513 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:41.136548 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":64,"threads_started":1}
I20260806 15:06:41.196049 26099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:41.195858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.66:47161)
0806 15:06:41.195983 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:41.196012 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":69,"threads_started":1}
I20260806 15:06:41.322088 26100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:41.321886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.67:51509)
0806 15:06:41.322016 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:41.322047 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":61,"threads_started":1}
I20260806 15:06:42.137209 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:42.137005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.65:43529)
0806 15:06:42.137129 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:42.137162 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":65,"threads_started":1}
I20260806 15:06:42.196604 26104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:42.196402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.66:53579)
0806 15:06:42.196533 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:42.196562 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":68,"threads_started":1}
I20260806 15:06:42.322619 26105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:42.322439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.67:33547)
0806 15:06:42.322546 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:42.322580 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":59,"threads_started":1}
I20260806 15:06:43.137815 26107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:43.137609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.65:58349)
0806 15:06:43.137734 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:43.137769 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260806 15:06:43.197162 26108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:43.196982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.66:35275)
0806 15:06:43.197093 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:43.197124 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":61,"threads_started":1}
I20260806 15:06:43.323127 26112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:43.322958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.67:58789)
0806 15:06:43.323058 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:43.323087 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":57,"threads_started":1}
I20260806 15:06:44.138437 26113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:44.138247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.65:60537)
0806 15:06:44.138358 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:44.138390 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":59,"threads_started":1}
I20260806 15:06:44.197655 26114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:44.197485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.66:56087)
0806 15:06:44.197585 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:44.197617 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":55,"threads_started":1}
I20260806 15:06:44.294451 26115 raft_consensus.cc:482] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:06:44.294543 26115 raft_consensus.cc:504] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } } peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } }
I20260806 15:06:44.294697 26115 leader_election.cc:288] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ea5947f4e19b4c6197aa84d31b44616c (127.0.236.67:37451), 9f2c60fb4db442c1b269ed12633570ee (127.0.236.66:37857)
W20260806 15:06:44.295025 25481 leader_election.cc:341] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9f2c60fb4db442c1b269ed12633570ee (127.0.236.66:37857): Not found: Tablet not found: 593cafd5812143b7ae5cf9113c56aec3
W20260806 15:06:44.295115 25485 leader_election.cc:341] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ea5947f4e19b4c6197aa84d31b44616c (127.0.236.67:37451): Not found: Tablet not found: 593cafd5812143b7ae5cf9113c56aec3
I20260806 15:06:44.295185 25485 leader_election.cc:302] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4f9b8139a434eada46943ee89d6cc01; no voters: 9f2c60fb4db442c1b269ed12633570ee, ea5947f4e19b4c6197aa84d31b44616c
I20260806 15:06:44.295260 26115 raft_consensus.cc:2726] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 15:06:44.323688 26117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:44.323506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.67:35497)
0806 15:06:44.323617 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:44.323651 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":66,"threads_started":1}
I20260806 15:06:45.139017 26119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:45.138821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.65:33809)
0806 15:06:45.138934 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:45.138969 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":60,"threads_started":1}
I20260806 15:06:45.198206 26120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:45.198013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.66:40273)
0806 15:06:45.198134 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:45.198169 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
I20260806 15:06:45.324227 26121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:45.324025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.67:50071)
0806 15:06:45.324154 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:45.324189 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":75,"threads_started":1}
I20260806 15:06:46.139572 26122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:46.139378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.65:37459)
0806 15:06:46.139495 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:46.139526 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":72,"threads_started":1}
I20260806 15:06:46.198738 26123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:46.198553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.66:33629)
0806 15:06:46.198668 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:46.198700 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":67,"threads_started":1}
I20260806 15:06:46.324785 26125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:46.324593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.67:44761)
0806 15:06:46.324710 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:46.324744 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":70,"threads_started":1}
I20260806 15:06:47.140136 26127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:47.139943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.65:35175)
0806 15:06:47.140061 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:47.140091 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
I20260806 15:06:47.199240 26128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:47.199068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.66:34791)
0806 15:06:47.199178 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:47.199206 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":59,"threads_started":1}
I20260806 15:06:47.325318 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:47.325116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.67:40289)
0806 15:06:47.325246 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:47.325280 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":66,"threads_started":1}
I20260806 15:06:47.705426 26133 raft_consensus.cc:482] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:06:47.705528 26133 raft_consensus.cc:504] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5947f4e19b4c6197aa84d31b44616c" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37451 } } peers { permanent_uuid: "9f2c60fb4db442c1b269ed12633570ee" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37857 } } peers { permanent_uuid: "c4f9b8139a434eada46943ee89d6cc01" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 40075 } }
I20260806 15:06:47.705665 26133 leader_election.cc:288] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ea5947f4e19b4c6197aa84d31b44616c (127.0.236.67:37451), 9f2c60fb4db442c1b269ed12633570ee (127.0.236.66:37857)
W20260806 15:06:47.706028 25485 leader_election.cc:341] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ea5947f4e19b4c6197aa84d31b44616c (127.0.236.67:37451): Not found: Tablet not found: 593cafd5812143b7ae5cf9113c56aec3
W20260806 15:06:47.706118 25481 leader_election.cc:341] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9f2c60fb4db442c1b269ed12633570ee (127.0.236.66:37857): Not found: Tablet not found: 593cafd5812143b7ae5cf9113c56aec3
I20260806 15:06:47.706176 25481 leader_election.cc:302] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4f9b8139a434eada46943ee89d6cc01; no voters: 9f2c60fb4db442c1b269ed12633570ee, ea5947f4e19b4c6197aa84d31b44616c
I20260806 15:06:47.706255 26133 raft_consensus.cc:2726] T 593cafd5812143b7ae5cf9113c56aec3 P c4f9b8139a434eada46943ee89d6cc01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 15:06:48.140687 26134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:48.140483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.65:42811)
0806 15:06:48.140608 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:48.140639 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260806 15:06:48.199788 26135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:48.199590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.66:39411)
0806 15:06:48.199714 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:48.199748 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
I20260806 15:06:48.325946 26136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:48.325730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.67:43459)
0806 15:06:48.325853 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:48.325888 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":71,"threads_started":1}
I20260806 15:06:49.141319 26139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:49.141126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.65:35097)
0806 15:06:49.141242 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:49.141273 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":56,"threads_started":1}
I20260806 15:06:49.200412 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:49.200196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.66:47231)
0806 15:06:49.200331 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:49.200377 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":72,"threads_started":1}
I20260806 15:06:49.325717 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 25400
I20260806 15:06:49.329978 26142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:49.328736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:39435 (local address 127.0.236.67:48003)
0806 15:06:49.329872 (+ 1136us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:49.329927 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.126:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1121,"thread_start_us":1057,"threads_started":1}
I20260806 15:06:49.335062 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 25666
I20260806 15:06:49.345695 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 25868
2026-08-06T15:06:49Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58167 ms)
[----------] 30 tests from SecurityITest (228555 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:06:49 dist-test-slave-tc2s krb5kdc[26145](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:06:49 dist-test-slave-tc2s krb5kdc[26145](info): set up 2 sockets
Aug 06 15:06:49 dist-test-slave-tc2s krb5kdc[26145](info): commencing operation
krb5kdc: starting...
W20260806 15:06:50.450214 945 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:06:50 dist-test-slave-tc2s krb5kdc[26145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:06:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:06:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:06:50.560011 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:43449
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:41855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:43449
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:06:50.638365 26172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:50.638571 26172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:50.638599 26172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:50.639717 26172 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:06:50.639758 26172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:50.639777 26172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:50.639792 26172 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:06:50.639808 26172 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:06:50.641278 26172 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:41855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:43449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:43449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:50.641520 26172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:50.641712 26172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:50.646263 26183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:50.646284 26172 server_base.cc:1034] running on GCE node
W20260806 15:06:50.646252 26190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:50.646250 26185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:50.646649 26172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:50.646886 26172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:50.648021 26172 hybrid_clock.cc:648] HybridClock initialized: now 1786028810648006 us; error 30 us; skew 500 ppm
Aug 06 15:06:50 dist-test-slave-tc2s krb5kdc[26145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028810, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:06:50.650326 26172 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:06:50.650676 26172 webserver.cc:460] Webserver started at http://127.0.236.126:42125/ using document root <none> and password file <none>
I20260806 15:06:50.650846 26172 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:50.650892 26172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:50.650979 26172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:06:50.651898 26172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "6e9a9a4ec8ea4bdd975225e4bfff01b5"
format_stamp: "Formatted at 2026-08-06 15:06:50 on dist-test-slave-tc2s"
server_key: "66154b558bab92d75f84647f22374213"
server_key_iv: "58256a8cd2c54c15f5d823d6c6758c8f"
server_key_version: "encryptionkey@0"
I20260806 15:06:50.652179 26172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "6e9a9a4ec8ea4bdd975225e4bfff01b5"
format_stamp: "Formatted at 2026-08-06 15:06:50 on dist-test-slave-tc2s"
server_key: "66154b558bab92d75f84647f22374213"
server_key_iv: "58256a8cd2c54c15f5d823d6c6758c8f"
server_key_version: "encryptionkey@0"
I20260806 15:06:50.653301 26172 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 15:06:50.653887 26198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:50.654066 26172 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:50.654147 26172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "6e9a9a4ec8ea4bdd975225e4bfff01b5"
format_stamp: "Formatted at 2026-08-06 15:06:50 on dist-test-slave-tc2s"
server_key: "66154b558bab92d75f84647f22374213"
server_key_iv: "58256a8cd2c54c15f5d823d6c6758c8f"
server_key_version: "encryptionkey@0"
I20260806 15:06:50.654212 26172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:50.676221 26172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:50.677759 26172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:50.677883 26172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:50.681581 26172 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:43449
I20260806 15:06:50.681625 26278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:43449 every 8 connection(s)
I20260806 15:06:50.681943 26172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:06:50.682752 26279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:50.684229 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 26172
I20260806 15:06:50.684331 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:06:50.684512 945 external_mini_cluster.cc:1428] Setting key 4c3f617fa181b8fd75ae4e55081d6839
I20260806 15:06:50.684953 26279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e9a9a4ec8ea4bdd975225e4bfff01b5: Bootstrap starting.
I20260806 15:06:50.685537 26279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e9a9a4ec8ea4bdd975225e4bfff01b5: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:50.685797 26279 log.cc:824] T 00000000000000000000000000000000 P 6e9a9a4ec8ea4bdd975225e4bfff01b5: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:50.686460 26279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e9a9a4ec8ea4bdd975225e4bfff01b5: No bootstrap required, opened a new log
I20260806 15:06:50.688149 26279 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e9a9a4ec8ea4bdd975225e4bfff01b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9a9a4ec8ea4bdd975225e4bfff01b5" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43449 } }
I20260806 15:06:50.688274 26279 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e9a9a4ec8ea4bdd975225e4bfff01b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:50.688305 26279 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e9a9a4ec8ea4bdd975225e4bfff01b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e9a9a4ec8ea4bdd975225e4bfff01b5, State: Initialized, Role: FOLLOWER
I20260806 15:06:50.688424 26279 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e9a9a4ec8ea4bdd975225e4bfff01b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9a9a4ec8ea4bdd975225e4bfff01b5" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43449 } }
I20260806 15:06:50.688498 26279 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e9a9a4ec8ea4bdd975225e4bfff01b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:06:50.688544 26279 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e9a9a4ec8ea4bdd975225e4bfff01b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:06:50.688589 26279 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e9a9a4ec8ea4bdd975225e4bfff01b5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:50.689066 26279 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e9a9a4ec8ea4bdd975225e4bfff01b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9a9a4ec8ea4bdd975225e4bfff01b5" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43449 } }
I20260806 15:06:50.689162 26279 leader_election.cc:302] T 00000000000000000000000000000000 P 6e9a9a4ec8ea4bdd975225e4bfff01b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e9a9a4ec8ea4bdd975225e4bfff01b5; no voters:
I20260806 15:06:50.689313 26279 leader_election.cc:288] T 00000000000000000000000000000000 P 6e9a9a4ec8ea4bdd975225e4bfff01b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:06:50.689339 26285 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e9a9a4ec8ea4bdd975225e4bfff01b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:06:50.689443 26285 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e9a9a4ec8ea4bdd975225e4bfff01b5 [term 1 LEADER]: Becoming Leader. State: Replica: 6e9a9a4ec8ea4bdd975225e4bfff01b5, State: Running, Role: LEADER
I20260806 15:06:50.689579 26285 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e9a9a4ec8ea4bdd975225e4bfff01b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9a9a4ec8ea4bdd975225e4bfff01b5" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43449 } }
I20260806 15:06:50.689570 26279 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e9a9a4ec8ea4bdd975225e4bfff01b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:06:50.689957 26288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e9a9a4ec8ea4bdd975225e4bfff01b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e9a9a4ec8ea4bdd975225e4bfff01b5. Latest consensus state: current_term: 1 leader_uuid: "6e9a9a4ec8ea4bdd975225e4bfff01b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9a9a4ec8ea4bdd975225e4bfff01b5" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43449 } } }
I20260806 15:06:50.690150 26288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e9a9a4ec8ea4bdd975225e4bfff01b5 [sys.catalog]: This master's current role is: LEADER
I20260806 15:06:50.690092 26286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e9a9a4ec8ea4bdd975225e4bfff01b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e9a9a4ec8ea4bdd975225e4bfff01b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9a9a4ec8ea4bdd975225e4bfff01b5" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43449 } } }
I20260806 15:06:50.690377 26286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e9a9a4ec8ea4bdd975225e4bfff01b5 [sys.catalog]: This master's current role is: LEADER
Aug 06 15:06:50 dist-test-slave-tc2s krb5kdc[26145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:06:50.691027 26290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:06:50.691534 26290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:06:50.693167 26290 catalog_manager.cc:1347] Generated new cluster ID: a54c5f21655d43b48afffe842d6fb797
I20260806 15:06:50.693215 26290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:06:50.694645 26282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:50.684962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55914 (local address 127.0.236.126:43449)
0806 15:06:50.685117 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:50.685126 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:50.685148 (+ 22us) server_negotiation.cc:407] Connection header received
0806 15:06:50.685542 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:50.685548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:50.685594 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:50.685726 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:50.686342 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:50.687232 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:50.687947 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:50.688363 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:50.688451 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:50.690984 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:50.690988 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:50.690989 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:06:50.691000 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:50.692699 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:50.693063 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:50.693066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:50.693066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:50.693171 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:50.693409 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:50.693410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:50.693410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:50.693699 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:50.693828 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:50.694301 (+ 473us) server_negotiation.cc:299] Negotiation successful
0806 15:06:50.694437 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":63,"threads_started":1}
I20260806 15:06:50.700105 26290 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:06:50.700791 26290 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:06:50.703974 26290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e9a9a4ec8ea4bdd975225e4bfff01b5: Generated new TSK 0
I20260806 15:06:50.704118 26290 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.236.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
I20260806 15:06:50.727867 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:0
--local_ip_for_outbound_sockets=127.0.236.65
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:43449
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:41855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:06:50.805764 26320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:50.805991 26320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:50.806017 26320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:50.807133 26320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:50.807171 26320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:50.807195 26320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:06:50.808595 26320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:41855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:43449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:50.808794 26320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:50.808997 26320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:50.813514 26327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:50.813534 26325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:50.813627 26320 server_base.cc:1034] running on GCE node
W20260806 15:06:50.813529 26329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:50.813860 26320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:50.814098 26320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:50.815243 26320 hybrid_clock.cc:648] HybridClock initialized: now 1786028810815229 us; error 29 us; skew 500 ppm
Aug 06 15:06:50 dist-test-slave-tc2s krb5kdc[26145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028810, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:06:50.817427 26320 init.cc:377] Logged in from keytab as kudu/127.0.236.65@KRBTEST.COM (short username kudu)
I20260806 15:06:50.817811 26320 webserver.cc:460] Webserver started at http://127.0.236.65:42777/ using document root <none> and password file <none>
I20260806 15:06:50.818019 26320 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:50.818063 26320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:50.818151 26320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:06:50.819051 26320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-0/data/instance:
uuid: "e47ea127efe04b2481a7ccdcc1ff77a9"
format_stamp: "Formatted at 2026-08-06 15:06:50 on dist-test-slave-tc2s"
server_key: "4425118bc8793084e18013ca9fc50530"
server_key_iv: "359e3e42d6031af7bca3e076977998f0"
server_key_version: "encryptionkey@0"
I20260806 15:06:50.819317 26320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-0/wal/instance:
uuid: "e47ea127efe04b2481a7ccdcc1ff77a9"
format_stamp: "Formatted at 2026-08-06 15:06:50 on dist-test-slave-tc2s"
server_key: "4425118bc8793084e18013ca9fc50530"
server_key_iv: "359e3e42d6031af7bca3e076977998f0"
server_key_version: "encryptionkey@0"
I20260806 15:06:50.820463 26320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:06:50.821053 26337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:50.821195 26320 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:06:50.821269 26320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "e47ea127efe04b2481a7ccdcc1ff77a9"
format_stamp: "Formatted at 2026-08-06 15:06:50 on dist-test-slave-tc2s"
server_key: "4425118bc8793084e18013ca9fc50530"
server_key_iv: "359e3e42d6031af7bca3e076977998f0"
server_key_version: "encryptionkey@0"
I20260806 15:06:50.821331 26320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:50.831096 26320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:50.832580 26320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:50.832691 26320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:50.832878 26320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:06:50.833134 26320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:06:50.833165 26320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:50.833184 26320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:06:50.833206 26320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:50.838536 26320 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:36289
I20260806 15:06:50.838591 26503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:36289 every 8 connection(s)
I20260806 15:06:50.838840 26320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:06:50.841979 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 26320
I20260806 15:06:50.842079 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-0/wal/instance
I20260806 15:06:50.842247 945 external_mini_cluster.cc:1428] Setting key 6e0f3ba1e2531aaecbaa39e0b5ef2f1a
Aug 06 15:06:50 dist-test-slave-tc2s krb5kdc[26145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028810, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:06:50.847463 26282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:50.839443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:49547 (local address 127.0.236.126:43449)
0806 15:06:50.839486 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:50.839487 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:50.839668 (+ 181us) server_negotiation.cc:407] Connection header received
0806 15:06:50.840467 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:50.840468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:50.840476 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:50.840531 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:50.841026 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:50.841504 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:50.842573 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:50.842797 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:50.842850 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:50.844888 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:50.844890 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:50.844890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:06:50.844898 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:50.846184 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:50.846593 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:50.846593 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:50.846594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:50.846639 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:50.846906 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:50.846906 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:50.846907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:50.846993 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:50.847044 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:50.847338 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 15:06:50.847401 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 15:06:50.847681 26507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:50.839402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43449 (local address 127.0.236.65:49547)
0806 15:06:50.839557 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:50.839595 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:50.840389 (+ 794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:50.840580 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:50.840584 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:50.840798 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:06:50.840965 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:50.840968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:50.841564 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:50.841565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:50.842499 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:50.842501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:50.842875 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:50.842876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:50.843004 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:50.843430 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:06:50.843439 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:06:50.844800 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:06:50.846254 (+ 1454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:50.846262 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:50.846264 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:50.846529 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:50.846687 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:50.846688 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:50.846688 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:50.846833 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:50.847052 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:06:50.847060 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:06:50.847214 (+ 154us) client_negotiation.cc:769] Sending connection context
0806 15:06:50.847322 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 15:06:50.847477 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
I20260806 15:06:50.848182 26504 heartbeater.cc:347] Connected to a master server at 127.0.236.126:43449
I20260806 15:06:50.848272 26504 heartbeater.cc:464] Registering TS with master...
I20260806 15:06:50.848448 26504 heartbeater.cc:511] Master 127.0.236.126:43449 requested a full tablet report, sending...
I20260806 15:06:50.848968 26224 ts_manager.cc:194] Registered new tserver with Master: e47ea127efe04b2481a7ccdcc1ff77a9 (127.0.236.65:36289)
I20260806 15:06:50.849820 26224 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.65@KRBTEST.COM'} at 127.0.236.65:49547
WARNING: no policy specified for kudu/127.0.236.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
I20260806 15:06:50.865774 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:0
--local_ip_for_outbound_sockets=127.0.236.66
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:43449
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:41855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:06:50.942932 26511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:50.943113 26511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:50.943143 26511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:50.944232 26511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:50.944269 26511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:50.944303 26511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:06:50.945727 26511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:41855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:43449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:50.945955 26511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:50.946147 26511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:50.950611 26521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:50.950623 26518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:50.950758 26511 server_base.cc:1034] running on GCE node
W20260806 15:06:50.950623 26517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:50.951025 26511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:50.951251 26511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:50.952380 26511 hybrid_clock.cc:648] HybridClock initialized: now 1786028810952373 us; error 26 us; skew 500 ppm
Aug 06 15:06:50 dist-test-slave-tc2s krb5kdc[26145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028810, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:06:50.954694 26511 init.cc:377] Logged in from keytab as kudu/127.0.236.66@KRBTEST.COM (short username kudu)
I20260806 15:06:50.955070 26511 webserver.cc:460] Webserver started at http://127.0.236.66:40929/ using document root <none> and password file <none>
I20260806 15:06:50.955251 26511 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:50.955296 26511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:50.955416 26511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:06:50.956378 26511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-1/data/instance:
uuid: "37b0d72ae8b64a48ac9a4d12f024b65d"
format_stamp: "Formatted at 2026-08-06 15:06:50 on dist-test-slave-tc2s"
server_key: "ff75bbb7a9734ba64d634ea6dffbcc1f"
server_key_iv: "709d4974238841b7e7926b1ffcbe6270"
server_key_version: "encryptionkey@0"
I20260806 15:06:50.956714 26511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-1/wal/instance:
uuid: "37b0d72ae8b64a48ac9a4d12f024b65d"
format_stamp: "Formatted at 2026-08-06 15:06:50 on dist-test-slave-tc2s"
server_key: "ff75bbb7a9734ba64d634ea6dffbcc1f"
server_key_iv: "709d4974238841b7e7926b1ffcbe6270"
server_key_version: "encryptionkey@0"
I20260806 15:06:50.957975 26511 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:06:50.958612 26531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:50.958750 26511 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:06:50.958830 26511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "37b0d72ae8b64a48ac9a4d12f024b65d"
format_stamp: "Formatted at 2026-08-06 15:06:50 on dist-test-slave-tc2s"
server_key: "ff75bbb7a9734ba64d634ea6dffbcc1f"
server_key_iv: "709d4974238841b7e7926b1ffcbe6270"
server_key_version: "encryptionkey@0"
I20260806 15:06:50.958895 26511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:50.972519 26511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:50.974082 26511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:50.974210 26511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:50.974403 26511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:06:50.974673 26511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:06:50.974706 26511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:50.974732 26511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:06:50.974749 26511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:50.979943 26511 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:32875
I20260806 15:06:50.979990 26707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:32875 every 8 connection(s)
I20260806 15:06:50.980242 26511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
Aug 06 15:06:50 dist-test-slave-tc2s krb5kdc[26145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028810, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:06:50.988364 26282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:50.980861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:50499 (local address 127.0.236.126:43449)
0806 15:06:50.980916 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:50.980918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:50.981110 (+ 192us) server_negotiation.cc:407] Connection header received
0806 15:06:50.981982 (+ 872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:50.981983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:50.981996 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:50.982048 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:50.982618 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:50.983117 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:50.984067 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:50.984271 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:50.984326 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:50.986140 (+ 1814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:50.986142 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:50.986142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:06:50.986149 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:50.987171 (+ 1022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:50.987517 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:50.987518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:50.987518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:50.987575 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:50.987780 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:50.987781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:50.987781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:50.987892 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:50.987950 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:50.988192 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 15:06:50.988276 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 15:06:50.988492 26711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:50.980834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43449 (local address 127.0.236.66:50499)
0806 15:06:50.981003 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:50.981036 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:50.981852 (+ 816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:50.982110 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:50.982114 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:50.982347 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:06:50.982526 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:50.982529 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:50.983200 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:50.983201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:50.983980 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:50.983982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:50.984340 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:50.984341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:50.984458 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:50.984827 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:06:50.984835 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:06:50.986053 (+ 1218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:06:50.987241 (+ 1188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:50.987248 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:50.987250 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:50.987451 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:50.987628 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:50.987629 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:50.987629 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:50.987724 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:50.987947 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:06:50.987953 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:06:50.988076 (+ 123us) client_negotiation.cc:769] Sending connection context
0806 15:06:50.988160 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 15:06:50.988301 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":82,"threads_started":1}
I20260806 15:06:50.988879 26708 heartbeater.cc:347] Connected to a master server at 127.0.236.126:43449
I20260806 15:06:50.988956 26708 heartbeater.cc:464] Registering TS with master...
I20260806 15:06:50.989104 26708 heartbeater.cc:511] Master 127.0.236.126:43449 requested a full tablet report, sending...
I20260806 15:06:50.989460 26224 ts_manager.cc:194] Registered new tserver with Master: 37b0d72ae8b64a48ac9a4d12f024b65d (127.0.236.66:32875)
I20260806 15:06:50.989789 26224 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.66@KRBTEST.COM'} at 127.0.236.66:50499
I20260806 15:06:50.990012 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 26511
I20260806 15:06:50.990129 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-1/wal/instance
I20260806 15:06:50.990355 945 external_mini_cluster.cc:1428] Setting key d55f919d8359618c6749648cf5d1e635
WARNING: no policy specified for kudu/127.0.236.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
I20260806 15:06:51.012428 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:0
--local_ip_for_outbound_sockets=127.0.236.67
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:43449
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:41855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:06:51.090420 26714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:51.090626 26714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:51.090648 26714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:51.091796 26714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:51.091836 26714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:51.091881 26714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:06:51.093303 26714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:41855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:43449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:51.093500 26714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:51.093684 26714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:51.098258 26724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:51.098273 26721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:51.098356 26714 server_base.cc:1034] running on GCE node
W20260806 15:06:51.098317 26720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:51.098644 26714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:51.098862 26714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:51.099995 26714 hybrid_clock.cc:648] HybridClock initialized: now 1786028811099981 us; error 27 us; skew 500 ppm
Aug 06 15:06:51 dist-test-slave-tc2s krb5kdc[26145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028811, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:06:51.102301 26714 init.cc:377] Logged in from keytab as kudu/127.0.236.67@KRBTEST.COM (short username kudu)
I20260806 15:06:51.102694 26714 webserver.cc:460] Webserver started at http://127.0.236.67:34593/ using document root <none> and password file <none>
I20260806 15:06:51.102857 26714 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:51.102895 26714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:51.102981 26714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:06:51.103868 26714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-2/data/instance:
uuid: "5b5ea9d012444d508c0a3d6bcc6d5085"
format_stamp: "Formatted at 2026-08-06 15:06:51 on dist-test-slave-tc2s"
server_key: "2231dda2580c5ea60f371d09a5e8ed2f"
server_key_iv: "a43d63759f9c469c84d5af28bbd767f3"
server_key_version: "encryptionkey@0"
I20260806 15:06:51.104137 26714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-2/wal/instance:
uuid: "5b5ea9d012444d508c0a3d6bcc6d5085"
format_stamp: "Formatted at 2026-08-06 15:06:51 on dist-test-slave-tc2s"
server_key: "2231dda2580c5ea60f371d09a5e8ed2f"
server_key_iv: "a43d63759f9c469c84d5af28bbd767f3"
server_key_version: "encryptionkey@0"
I20260806 15:06:51.105206 26714 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 15:06:51.105778 26732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:51.105950 26714 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:51.106034 26714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "5b5ea9d012444d508c0a3d6bcc6d5085"
format_stamp: "Formatted at 2026-08-06 15:06:51 on dist-test-slave-tc2s"
server_key: "2231dda2580c5ea60f371d09a5e8ed2f"
server_key_iv: "a43d63759f9c469c84d5af28bbd767f3"
server_key_version: "encryptionkey@0"
I20260806 15:06:51.106086 26714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:51.116667 26714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:51.118233 26714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:51.118335 26714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:51.118502 26714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:06:51.118732 26714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:06:51.118762 26714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:51.118779 26714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:06:51.118795 26714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:51.124366 26714 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:46725
I20260806 15:06:51.124432 26903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:46725 every 8 connection(s)
I20260806 15:06:51.124676 26714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
I20260806 15:06:51.126243 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 26714
I20260806 15:06:51.126366 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-2/wal/instance
I20260806 15:06:51.126565 945 external_mini_cluster.cc:1428] Setting key 081bf7887226748c251d37238fc2c705
Aug 06 15:06:51 dist-test-slave-tc2s krb5kdc[26145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028811, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:06:51.134887 26282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.125314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:50841 (local address 127.0.236.126:43449)
0806 15:06:51.125354 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:51.125356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:51.125536 (+ 180us) server_negotiation.cc:407] Connection header received
0806 15:06:51.126411 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:51.126413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:51.126425 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:51.126478 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:51.127206 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.127746 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.128677 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.128912 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.128967 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.131802 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:51.131804 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:51.131804 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:06:51.131814 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:51.133431 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:51.133915 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:51.133916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:51.133917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:51.133992 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:51.134253 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:51.134254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:51.134255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:51.134387 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:51.134447 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:51.134708 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 15:06:51.134798 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:06:51.135021 26906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.125268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43449 (local address 127.0.236.67:50841)
0806 15:06:51.125424 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:51.125465 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:51.126304 (+ 839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:51.126543 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:51.126546 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:51.126862 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:06:51.127126 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.127132 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.127816 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.127817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.128591 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.128593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.128985 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.128986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.129106 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.129628 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:06:51.129638 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:06:51.131660 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:06:51.133538 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:51.133546 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:51.133548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:51.133827 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:51.134053 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:51.134054 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:51.134055 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:51.134159 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:51.134461 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:06:51.134468 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:06:51.134592 (+ 124us) client_negotiation.cc:769] Sending connection context
0806 15:06:51.134676 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 15:06:51.134822 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":97,"threads_started":1}
I20260806 15:06:51.137980 26904 heartbeater.cc:347] Connected to a master server at 127.0.236.126:43449
I20260806 15:06:51.138079 26904 heartbeater.cc:464] Registering TS with master...
I20260806 15:06:51.138250 26904 heartbeater.cc:511] Master 127.0.236.126:43449 requested a full tablet report, sending...
I20260806 15:06:51.138614 26224 ts_manager.cc:194] Registered new tserver with Master: 5b5ea9d012444d508c0a3d6bcc6d5085 (127.0.236.67:46725)
I20260806 15:06:51.139101 26224 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.67@KRBTEST.COM'} at 127.0.236.67:50841
I20260806 15:06:51.148903 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:06:51.155720 26282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.150218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55920 (local address 127.0.236.126:43449)
0806 15:06:51.150269 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:51.150271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:51.150364 (+ 93us) server_negotiation.cc:407] Connection header received
0806 15:06:51.150421 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:51.150422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:51.150431 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:51.150482 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:51.150992 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.151458 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.152145 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.152341 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.152392 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.153447 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:51.153448 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:51.153448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:06:51.153455 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:51.154695 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:51.155016 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:51.155017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:51.155017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:51.155069 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:51.155227 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:51.155227 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:51.155228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:51.155365 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:51.155462 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:51.155564 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 15:06:51.155636 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 15:06:51.157173 26224 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55920:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:06:51.158793 26224 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:06:51.165863 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.160780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:46725 (local address 127.0.0.1:43238)
0806 15:06:51.161201 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:51.161240 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:51.161321 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:51.162315 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:51.162316 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:51.162318 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:51.162355 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.162356 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.163499 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.163500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.164629 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.164632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.165567 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.165568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.165667 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.165670 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:06:51.165731 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 15:06:51.165792 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":6,"thread_start_us":53,"threads_started":1}
I20260806 15:06:51.166098 26922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.161041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54920 (local address 127.0.236.65:36289)
0806 15:06:51.161252 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:51.161257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:51.161505 (+ 248us) server_negotiation.cc:407] Connection header received
0806 15:06:51.161554 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:51.161557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:51.161572 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:51.161645 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:51.161935 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.163012 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.164597 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.165397 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.165466 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.165523 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:51.165783 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 15:06:51.165826 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":66,"threads_started":1}
I20260806 15:06:51.166147 26915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.160806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:36289 (local address 127.0.0.1:54920)
0806 15:06:51.161421 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:51.161429 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:51.161489 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:51.161766 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:51.161769 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:51.161771 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:51.161835 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.161837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.163128 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.163129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.164496 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.164499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.165560 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.165561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.165655 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.165668 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 15:06:51.165733 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 15:06:51.166086 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"mutex_wait_us":367,"thread_start_us":84,"threads_started":1}
I20260806 15:06:51.166383 26921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.160981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54042 (local address 127.0.236.66:32875)
0806 15:06:51.161160 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:51.161164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:51.161304 (+ 140us) server_negotiation.cc:407] Connection header received
0806 15:06:51.161401 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:51.161404 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:51.161418 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:51.161600 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:51.161931 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.162976 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.164725 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.165418 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.165474 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.165546 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:51.166265 (+ 719us) server_negotiation.cc:299] Negotiation successful
0806 15:06:51.166309 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":68,"threads_started":1}
I20260806 15:06:51.166595 26917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.160780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:32875 (local address 127.0.0.1:54042)
0806 15:06:51.161201 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:51.161240 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:51.161319 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:51.161767 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:51.161770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:51.161772 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:51.161840 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.161842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.163084 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.163085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.164629 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.164632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.165611 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.165612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.166056 (+ 444us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.166061 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 15:06:51.166490 (+ 429us) client_negotiation.cc:240] Negotiation successful
0806 15:06:51.166534 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":47,"thread_start_us":46,"threads_started":1}
I20260806 15:06:51.166832 26411 tablet_service.cc:1467] Processing CreateTablet for tablet e54c9b4238494fe081387d0c110018f9 (DEFAULT_TABLE table=test-table [id=b90f502e9b2f470789c958fb3b243d2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:06:51.167121 26411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e54c9b4238494fe081387d0c110018f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:51.169618 26925 tablet_bootstrap.cc:492] T e54c9b4238494fe081387d0c110018f9 P e47ea127efe04b2481a7ccdcc1ff77a9: Bootstrap starting.
I20260806 15:06:51.170310 26925 tablet_bootstrap.cc:654] T e54c9b4238494fe081387d0c110018f9 P e47ea127efe04b2481a7ccdcc1ff77a9: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:51.170202 26595 tablet_service.cc:1467] Processing CreateTablet for tablet e54c9b4238494fe081387d0c110018f9 (DEFAULT_TABLE table=test-table [id=b90f502e9b2f470789c958fb3b243d2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:06:51.170321 26919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.160970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43238 (local address 127.0.236.67:46725)
0806 15:06:51.161141 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:51.161146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:51.161328 (+ 182us) server_negotiation.cc:407] Connection header received
0806 15:06:51.162066 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:51.162069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:51.162086 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:51.162250 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:51.162426 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.163396 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.164718 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.165417 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.165483 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.165529 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:51.170192 (+ 4663us) server_negotiation.cc:299] Negotiation successful
0806 15:06:51.170239 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":79,"threads_started":1}
I20260806 15:06:51.170538 26595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e54c9b4238494fe081387d0c110018f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:51.171066 26925 log.cc:824] T e54c9b4238494fe081387d0c110018f9 P e47ea127efe04b2481a7ccdcc1ff77a9: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:51.171159 26804 tablet_service.cc:1467] Processing CreateTablet for tablet e54c9b4238494fe081387d0c110018f9 (DEFAULT_TABLE table=test-table [id=b90f502e9b2f470789c958fb3b243d2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:06:51.171450 26804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e54c9b4238494fe081387d0c110018f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:51.171800 26925 tablet_bootstrap.cc:492] T e54c9b4238494fe081387d0c110018f9 P e47ea127efe04b2481a7ccdcc1ff77a9: No bootstrap required, opened a new log
I20260806 15:06:51.171862 26925 ts_tablet_manager.cc:1397] T e54c9b4238494fe081387d0c110018f9 P e47ea127efe04b2481a7ccdcc1ff77a9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 15:06:51.173774 26925 raft_consensus.cc:348] T e54c9b4238494fe081387d0c110018f9 P e47ea127efe04b2481a7ccdcc1ff77a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b5ea9d012444d508c0a3d6bcc6d5085" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46725 } } peers { permanent_uuid: "e47ea127efe04b2481a7ccdcc1ff77a9" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36289 } } peers { permanent_uuid: "37b0d72ae8b64a48ac9a4d12f024b65d" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 32875 } }
I20260806 15:06:51.173977 26925 raft_consensus.cc:374] T e54c9b4238494fe081387d0c110018f9 P e47ea127efe04b2481a7ccdcc1ff77a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:51.174021 26925 raft_consensus.cc:729] T e54c9b4238494fe081387d0c110018f9 P e47ea127efe04b2481a7ccdcc1ff77a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47ea127efe04b2481a7ccdcc1ff77a9, State: Initialized, Role: FOLLOWER
I20260806 15:06:51.174117 26928 tablet_bootstrap.cc:492] T e54c9b4238494fe081387d0c110018f9 P 37b0d72ae8b64a48ac9a4d12f024b65d: Bootstrap starting.
I20260806 15:06:51.174130 26925 consensus_queue.cc:260] T e54c9b4238494fe081387d0c110018f9 P e47ea127efe04b2481a7ccdcc1ff77a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b5ea9d012444d508c0a3d6bcc6d5085" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46725 } } peers { permanent_uuid: "e47ea127efe04b2481a7ccdcc1ff77a9" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36289 } } peers { permanent_uuid: "37b0d72ae8b64a48ac9a4d12f024b65d" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 32875 } }
I20260806 15:06:51.174386 26925 ts_tablet_manager.cc:1428] T e54c9b4238494fe081387d0c110018f9 P e47ea127efe04b2481a7ccdcc1ff77a9: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 15:06:51.174548 26504 heartbeater.cc:503] Master 127.0.236.126:43449 was elected leader, sending a full tablet report...
I20260806 15:06:51.175091 26928 tablet_bootstrap.cc:654] T e54c9b4238494fe081387d0c110018f9 P 37b0d72ae8b64a48ac9a4d12f024b65d: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:51.175412 26928 log.cc:824] T e54c9b4238494fe081387d0c110018f9 P 37b0d72ae8b64a48ac9a4d12f024b65d: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:51.176105 26928 tablet_bootstrap.cc:492] T e54c9b4238494fe081387d0c110018f9 P 37b0d72ae8b64a48ac9a4d12f024b65d: No bootstrap required, opened a new log
I20260806 15:06:51.176157 26928 ts_tablet_manager.cc:1397] T e54c9b4238494fe081387d0c110018f9 P 37b0d72ae8b64a48ac9a4d12f024b65d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:06:51.177471 26928 raft_consensus.cc:348] T e54c9b4238494fe081387d0c110018f9 P 37b0d72ae8b64a48ac9a4d12f024b65d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b5ea9d012444d508c0a3d6bcc6d5085" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46725 } } peers { permanent_uuid: "e47ea127efe04b2481a7ccdcc1ff77a9" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36289 } } peers { permanent_uuid: "37b0d72ae8b64a48ac9a4d12f024b65d" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 32875 } }
I20260806 15:06:51.177603 26928 raft_consensus.cc:374] T e54c9b4238494fe081387d0c110018f9 P 37b0d72ae8b64a48ac9a4d12f024b65d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:51.177623 26928 raft_consensus.cc:729] T e54c9b4238494fe081387d0c110018f9 P 37b0d72ae8b64a48ac9a4d12f024b65d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37b0d72ae8b64a48ac9a4d12f024b65d, State: Initialized, Role: FOLLOWER
I20260806 15:06:51.177738 26928 consensus_queue.cc:260] T e54c9b4238494fe081387d0c110018f9 P 37b0d72ae8b64a48ac9a4d12f024b65d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b5ea9d012444d508c0a3d6bcc6d5085" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46725 } } peers { permanent_uuid: "e47ea127efe04b2481a7ccdcc1ff77a9" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36289 } } peers { permanent_uuid: "37b0d72ae8b64a48ac9a4d12f024b65d" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 32875 } }
I20260806 15:06:51.177997 26708 heartbeater.cc:503] Master 127.0.236.126:43449 was elected leader, sending a full tablet report...
I20260806 15:06:51.178004 26928 ts_tablet_manager.cc:1428] T e54c9b4238494fe081387d0c110018f9 P 37b0d72ae8b64a48ac9a4d12f024b65d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:06:51.178081 26931 tablet_bootstrap.cc:492] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085: Bootstrap starting.
I20260806 15:06:51.178732 26931 tablet_bootstrap.cc:654] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:51.179239 26931 log.cc:824] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:51.179945 26931 tablet_bootstrap.cc:492] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085: No bootstrap required, opened a new log
I20260806 15:06:51.180013 26931 ts_tablet_manager.cc:1397] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:06:51.181864 26931 raft_consensus.cc:348] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b5ea9d012444d508c0a3d6bcc6d5085" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46725 } } peers { permanent_uuid: "e47ea127efe04b2481a7ccdcc1ff77a9" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36289 } } peers { permanent_uuid: "37b0d72ae8b64a48ac9a4d12f024b65d" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 32875 } }
I20260806 15:06:51.182053 26931 raft_consensus.cc:374] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:51.182093 26931 raft_consensus.cc:729] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b5ea9d012444d508c0a3d6bcc6d5085, State: Initialized, Role: FOLLOWER
I20260806 15:06:51.182202 26931 consensus_queue.cc:260] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b5ea9d012444d508c0a3d6bcc6d5085" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46725 } } peers { permanent_uuid: "e47ea127efe04b2481a7ccdcc1ff77a9" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36289 } } peers { permanent_uuid: "37b0d72ae8b64a48ac9a4d12f024b65d" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 32875 } }
I20260806 15:06:51.183063 26904 heartbeater.cc:503] Master 127.0.236.126:43449 was elected leader, sending a full tablet report...
I20260806 15:06:51.183049 26931 ts_tablet_manager.cc:1428] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 15:06:51.200421 26939 raft_consensus.cc:482] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:06:51.200510 26939 raft_consensus.cc:504] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b5ea9d012444d508c0a3d6bcc6d5085" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46725 } } peers { permanent_uuid: "e47ea127efe04b2481a7ccdcc1ff77a9" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36289 } } peers { permanent_uuid: "37b0d72ae8b64a48ac9a4d12f024b65d" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 32875 } }
I20260806 15:06:51.200773 26939 leader_election.cc:288] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e47ea127efe04b2481a7ccdcc1ff77a9 (127.0.236.65:36289), 37b0d72ae8b64a48ac9a4d12f024b65d (127.0.236.66:32875)
I20260806 15:06:51.203429 26921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.200967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:44481 (local address 127.0.236.66:32875)
0806 15:06:51.201000 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:51.201002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:51.201010 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:06:51.201079 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:51.201080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:51.201091 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:51.201134 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:51.201343 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.201806 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.202775 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.203151 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.203211 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.203219 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:51.203341 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 15:06:51.203372 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:06:51.203442 26922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.201114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:55053 (local address 127.0.236.65:36289)
0806 15:06:51.201149 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:51.201151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:51.201159 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:06:51.201179 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:51.201180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:51.201188 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:51.201242 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:51.201387 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.201821 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.202787 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.203154 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.203216 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.203223 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:51.203353 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 15:06:51.203385 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:06:51.203449 26906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.200903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:32875 (local address 127.0.236.67:44481)
0806 15:06:51.200962 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:51.200972 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:51.201008 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:51.201206 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:51.201207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:51.201208 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:51.201268 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.201271 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.201914 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.201915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.202694 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.202697 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.203219 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.203219 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.203284 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.203287 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:06:51.203331 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 15:06:51.203374 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":36}
I20260806 15:06:51.203656 26940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.200933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:36289 (local address 127.0.236.67:55053)
0806 15:06:51.201059 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:51.201068 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:51.201106 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:51.201244 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:51.201245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:51.201246 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:51.201300 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.201301 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.201908 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.201909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.202710 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.202712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.203222 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.203223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.203297 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.203300 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:06:51.203343 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 15:06:51.203381 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":36,"thread_start_us":70,"threads_started":1}
I20260806 15:06:51.203766 26630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e54c9b4238494fe081387d0c110018f9" candidate_uuid: "5b5ea9d012444d508c0a3d6bcc6d5085" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37b0d72ae8b64a48ac9a4d12f024b65d" is_pre_election: true
I20260806 15:06:51.203862 26630 raft_consensus.cc:2454] T e54c9b4238494fe081387d0c110018f9 P 37b0d72ae8b64a48ac9a4d12f024b65d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b5ea9d012444d508c0a3d6bcc6d5085 in term 0.
I20260806 15:06:51.204051 26733 leader_election.cc:302] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37b0d72ae8b64a48ac9a4d12f024b65d, 5b5ea9d012444d508c0a3d6bcc6d5085; no voters:
I20260806 15:06:51.204119 26442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e54c9b4238494fe081387d0c110018f9" candidate_uuid: "5b5ea9d012444d508c0a3d6bcc6d5085" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ea127efe04b2481a7ccdcc1ff77a9" is_pre_election: true
I20260806 15:06:51.204208 26939 raft_consensus.cc:2781] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:06:51.204241 26442 raft_consensus.cc:2454] T e54c9b4238494fe081387d0c110018f9 P e47ea127efe04b2481a7ccdcc1ff77a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b5ea9d012444d508c0a3d6bcc6d5085 in term 0.
I20260806 15:06:51.204272 26939 raft_consensus.cc:482] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:06:51.204303 26939 raft_consensus.cc:3037] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:51.204869 26939 raft_consensus.cc:504] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b5ea9d012444d508c0a3d6bcc6d5085" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46725 } } peers { permanent_uuid: "e47ea127efe04b2481a7ccdcc1ff77a9" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36289 } } peers { permanent_uuid: "37b0d72ae8b64a48ac9a4d12f024b65d" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 32875 } }
I20260806 15:06:51.204994 26939 leader_election.cc:288] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085 [CANDIDATE]: Term 1 election: Requested vote from peers e47ea127efe04b2481a7ccdcc1ff77a9 (127.0.236.65:36289), 37b0d72ae8b64a48ac9a4d12f024b65d (127.0.236.66:32875)
I20260806 15:06:51.205142 26630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e54c9b4238494fe081387d0c110018f9" candidate_uuid: "5b5ea9d012444d508c0a3d6bcc6d5085" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37b0d72ae8b64a48ac9a4d12f024b65d"
I20260806 15:06:51.205173 26442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e54c9b4238494fe081387d0c110018f9" candidate_uuid: "5b5ea9d012444d508c0a3d6bcc6d5085" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e47ea127efe04b2481a7ccdcc1ff77a9"
I20260806 15:06:51.205216 26630 raft_consensus.cc:3037] T e54c9b4238494fe081387d0c110018f9 P 37b0d72ae8b64a48ac9a4d12f024b65d [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:51.205236 26442 raft_consensus.cc:3037] T e54c9b4238494fe081387d0c110018f9 P e47ea127efe04b2481a7ccdcc1ff77a9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:51.205806 26630 raft_consensus.cc:2454] T e54c9b4238494fe081387d0c110018f9 P 37b0d72ae8b64a48ac9a4d12f024b65d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b5ea9d012444d508c0a3d6bcc6d5085 in term 1.
I20260806 15:06:51.205868 26442 raft_consensus.cc:2454] T e54c9b4238494fe081387d0c110018f9 P e47ea127efe04b2481a7ccdcc1ff77a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b5ea9d012444d508c0a3d6bcc6d5085 in term 1.
I20260806 15:06:51.205971 26733 leader_election.cc:302] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37b0d72ae8b64a48ac9a4d12f024b65d, 5b5ea9d012444d508c0a3d6bcc6d5085; no voters:
I20260806 15:06:51.206064 26939 raft_consensus.cc:2781] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:06:51.206203 26939 raft_consensus.cc:686] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085 [term 1 LEADER]: Becoming Leader. State: Replica: 5b5ea9d012444d508c0a3d6bcc6d5085, State: Running, Role: LEADER
I20260806 15:06:51.206281 26939 consensus_queue.cc:237] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b5ea9d012444d508c0a3d6bcc6d5085" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46725 } } peers { permanent_uuid: "e47ea127efe04b2481a7ccdcc1ff77a9" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36289 } } peers { permanent_uuid: "37b0d72ae8b64a48ac9a4d12f024b65d" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 32875 } }
I20260806 15:06:51.206969 26223 catalog_manager.cc:5495] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b5ea9d012444d508c0a3d6bcc6d5085 (127.0.236.67). New cstate: current_term: 1 leader_uuid: "5b5ea9d012444d508c0a3d6bcc6d5085" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b5ea9d012444d508c0a3d6bcc6d5085" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e47ea127efe04b2481a7ccdcc1ff77a9" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37b0d72ae8b64a48ac9a4d12f024b65d" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 32875 } health_report { overall_health: UNKNOWN } } }
I20260806 15:06:51.223032 26919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.220919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43240 (local address 127.0.236.67:46725)
0806 15:06:51.220965 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:51.220966 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:51.220974 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:06:51.221036 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:51.221037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:51.221054 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:51.221108 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:06:51.221228 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.221646 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.222384 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.222558 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.222609 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.222749 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:06:51.222835 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:06:51.222906 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:51.222941 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 15:06:51.222959 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 15:06:51.224400 26442 raft_consensus.cc:1261] T e54c9b4238494fe081387d0c110018f9 P e47ea127efe04b2481a7ccdcc1ff77a9 [term 1 FOLLOWER]: Refusing update from remote peer 5b5ea9d012444d508c0a3d6bcc6d5085: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:06:51.224393 26630 raft_consensus.cc:1261] T e54c9b4238494fe081387d0c110018f9 P 37b0d72ae8b64a48ac9a4d12f024b65d [term 1 FOLLOWER]: Refusing update from remote peer 5b5ea9d012444d508c0a3d6bcc6d5085: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:06:51.224695 26941 consensus_queue.cc:1035] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e47ea127efe04b2481a7ccdcc1ff77a9" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 36289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:06:51.224803 26939 consensus_queue.cc:1035] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37b0d72ae8b64a48ac9a4d12f024b65d" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 32875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:06:51.229126 26224 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:06:51.229203 26224 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:06:51.229866 26224 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e9a9a4ec8ea4bdd975225e4bfff01b5: Sending DeleteTablet for 3 replicas of tablet e54c9b4238494fe081387d0c110018f9
I20260806 15:06:51.230175 26804 tablet_service.cc:1514] Processing DeleteTablet for tablet e54c9b4238494fe081387d0c110018f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:06:51 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:43238
I20260806 15:06:51.230188 26411 tablet_service.cc:1514] Processing DeleteTablet for tablet e54c9b4238494fe081387d0c110018f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:06:51 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:54920
I20260806 15:06:51.230392 26595 tablet_service.cc:1514] Processing DeleteTablet for tablet e54c9b4238494fe081387d0c110018f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:06:51 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:54042
I20260806 15:06:51.230412 26956 tablet_replica.cc:330] T e54c9b4238494fe081387d0c110018f9 P e47ea127efe04b2481a7ccdcc1ff77a9: stopping tablet replica
I20260806 15:06:51.230506 26956 raft_consensus.cc:2229] T e54c9b4238494fe081387d0c110018f9 P e47ea127efe04b2481a7ccdcc1ff77a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:06:51.230633 26959 tablet_replica.cc:330] T e54c9b4238494fe081387d0c110018f9 P 37b0d72ae8b64a48ac9a4d12f024b65d: stopping tablet replica
I20260806 15:06:51.230607 26956 raft_consensus.cc:2258] T e54c9b4238494fe081387d0c110018f9 P e47ea127efe04b2481a7ccdcc1ff77a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:06:51.230849 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 26320
I20260806 15:06:51.230803 26959 raft_consensus.cc:2229] T e54c9b4238494fe081387d0c110018f9 P 37b0d72ae8b64a48ac9a4d12f024b65d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:06:51.230932 26958 tablet_replica.cc:330] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085: stopping tablet replica
I20260806 15:06:51.231020 26958 raft_consensus.cc:2229] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085 [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:06:51.231043 26959 raft_consensus.cc:2258] T e54c9b4238494fe081387d0c110018f9 P 37b0d72ae8b64a48ac9a4d12f024b65d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:06:51.231102 26958 raft_consensus.cc:2258] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:06:51.233354 26959 ts_tablet_manager.cc:1905] T e54c9b4238494fe081387d0c110018f9 P 37b0d72ae8b64a48ac9a4d12f024b65d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:06:51.233389 26958 ts_tablet_manager.cc:1905] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:06:51.234836 26959 ts_tablet_manager.cc:1918] T e54c9b4238494fe081387d0c110018f9 P 37b0d72ae8b64a48ac9a4d12f024b65d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:06:51.234854 26958 ts_tablet_manager.cc:1918] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:06:51.234915 26958 log.cc:1196] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/e54c9b4238494fe081387d0c110018f9
I20260806 15:06:51.235034 26959 log.cc:1196] T e54c9b4238494fe081387d0c110018f9 P 37b0d72ae8b64a48ac9a4d12f024b65d: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028580791223-945-0/minicluster-data/ts-1/wal/wals/e54c9b4238494fe081387d0c110018f9
I20260806 15:06:51.235172 26958 ts_tablet_manager.cc:1939] T e54c9b4238494fe081387d0c110018f9 P 5b5ea9d012444d508c0a3d6bcc6d5085: Deleting consensus metadata
I20260806 15:06:51.235519 26959 ts_tablet_manager.cc:1939] T e54c9b4238494fe081387d0c110018f9 P 37b0d72ae8b64a48ac9a4d12f024b65d: Deleting consensus metadata
I20260806 15:06:51.235585 26201 catalog_manager.cc:4841] TS 5b5ea9d012444d508c0a3d6bcc6d5085 (127.0.236.67:46725): tablet e54c9b4238494fe081387d0c110018f9 (table test-table [id=b90f502e9b2f470789c958fb3b243d2f]) successfully deleted
I20260806 15:06:51.236068 26200 catalog_manager.cc:4841] TS 37b0d72ae8b64a48ac9a4d12f024b65d (127.0.236.66:32875): tablet e54c9b4238494fe081387d0c110018f9 (table test-table [id=b90f502e9b2f470789c958fb3b243d2f]) successfully deleted
W20260806 15:06:51.244654 26202 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.236.65:36289 (error 108)
I20260806 15:06:51.245092 26915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.244981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:36289 (local address 127.0.0.1:54930)
0806 15:06:51.245027 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:51.245063 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.65:36289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 15:06:51.245189 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 26511
W20260806 15:06:51.255636 26202 catalog_manager.cc:4584] TS e47ea127efe04b2481a7ccdcc1ff77a9 (127.0.236.65:36289): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e54c9b4238494fe081387d0c110018f9: Network error: Client connection negotiation failed: client connection to 127.0.236.65:36289: connect: Connection refused (error 111)
I20260806 15:06:51.259775 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 26714
I20260806 15:06:51.270547 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 26172
2026-08-06T15:06:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1931 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T15:06:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:06:51Z Disabled control of system clock
I20260806 15:06:51.304458 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:43877
--webserver_interface=127.0.236.126
--webserver_port=0
--builtin_ntp_servers=127.0.236.84:43085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:43877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 15:06:51.384933 26966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:51.385118 26966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:51.385136 26966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:51.386251 26966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:06:51.386288 26966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:51.386300 26966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:51.386323 26966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:06:51.386339 26966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:06:51.387646 26966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:43085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:43877
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:43877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:51.387853 26966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:51.388105 26966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:51.392372 26972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:51.392401 26976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:51.392465 26973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:51.392630 26966 server_base.cc:1034] running on GCE node
I20260806 15:06:51.392874 26966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:51.393129 26966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:51.394259 26966 hybrid_clock.cc:648] HybridClock initialized: now 1786028811394245 us; error 29 us; skew 500 ppm
I20260806 15:06:51.395323 26966 webserver.cc:460] Webserver started at http://127.0.236.126:41705/ using document root <none> and password file <none>
I20260806 15:06:51.395634 26966 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:51.395694 26966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:51.395816 26966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:06:51.396806 26966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "5a30f807f23d49ccadc653d246623d74"
format_stamp: "Formatted at 2026-08-06 15:06:51 on dist-test-slave-tc2s"
server_key: "d55dd97520aa48b935e6a07c90c464f5"
server_key_iv: "bd48abd64e5238fb2a3757d6bd92d09b"
server_key_version: "encryptionkey@0"
I20260806 15:06:51.397118 26966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "5a30f807f23d49ccadc653d246623d74"
format_stamp: "Formatted at 2026-08-06 15:06:51 on dist-test-slave-tc2s"
server_key: "d55dd97520aa48b935e6a07c90c464f5"
server_key_iv: "bd48abd64e5238fb2a3757d6bd92d09b"
server_key_version: "encryptionkey@0"
I20260806 15:06:51.398355 26966 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:06:51.398983 26985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:51.399124 26966 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:06:51.399205 26966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "5a30f807f23d49ccadc653d246623d74"
format_stamp: "Formatted at 2026-08-06 15:06:51 on dist-test-slave-tc2s"
server_key: "d55dd97520aa48b935e6a07c90c464f5"
server_key_iv: "bd48abd64e5238fb2a3757d6bd92d09b"
server_key_version: "encryptionkey@0"
I20260806 15:06:51.399262 26966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:51.417028 26966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:51.417311 26966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:51.417443 26966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:51.421244 26966 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:43877
I20260806 15:06:51.421278 27062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:43877 every 8 connection(s)
I20260806 15:06:51.421581 26966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:06:51.422541 27063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:51.423623 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 26966
I20260806 15:06:51.423713 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:06:51.423877 945 external_mini_cluster.cc:1428] Setting key ff77f35f0a8062931fcc8a56baee4edf
I20260806 15:06:51.424939 27063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a30f807f23d49ccadc653d246623d74: Bootstrap starting.
I20260806 15:06:51.425503 27063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a30f807f23d49ccadc653d246623d74: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:51.425766 27063 log.cc:824] T 00000000000000000000000000000000 P 5a30f807f23d49ccadc653d246623d74: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:51.426414 27063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a30f807f23d49ccadc653d246623d74: No bootstrap required, opened a new log
I20260806 15:06:51.428067 27063 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a30f807f23d49ccadc653d246623d74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a30f807f23d49ccadc653d246623d74" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43877 } }
I20260806 15:06:51.428195 27063 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a30f807f23d49ccadc653d246623d74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:51.428227 27063 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a30f807f23d49ccadc653d246623d74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a30f807f23d49ccadc653d246623d74, State: Initialized, Role: FOLLOWER
I20260806 15:06:51.428355 27063 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a30f807f23d49ccadc653d246623d74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a30f807f23d49ccadc653d246623d74" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43877 } }
I20260806 15:06:51.428424 27063 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a30f807f23d49ccadc653d246623d74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:06:51.428454 27063 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a30f807f23d49ccadc653d246623d74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:06:51.428503 27063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a30f807f23d49ccadc653d246623d74 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:51.428933 27066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.424363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55916 (local address 127.0.236.126:43877)
0806 15:06:51.424557 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:51.424564 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:51.424587 (+ 23us) server_negotiation.cc:407] Connection header received
0806 15:06:51.424964 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:51.424970 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:51.425024 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:51.425156 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:51.425370 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.426325 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.427117 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.427535 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.427651 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.428234 (+ 583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:51.428239 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:51.428239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:06:51.428423 (+ 184us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:51.428470 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:06:51.428489 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:51.428546 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:51.428666 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 15:06:51.428731 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":81,"threads_started":1}
I20260806 15:06:51.429098 27063 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a30f807f23d49ccadc653d246623d74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a30f807f23d49ccadc653d246623d74" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43877 } }
I20260806 15:06:51.429200 27063 leader_election.cc:302] T 00000000000000000000000000000000 P 5a30f807f23d49ccadc653d246623d74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a30f807f23d49ccadc653d246623d74; no voters:
I20260806 15:06:51.429370 27063 leader_election.cc:288] T 00000000000000000000000000000000 P 5a30f807f23d49ccadc653d246623d74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:06:51.429404 27068 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a30f807f23d49ccadc653d246623d74 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:06:51.429558 27068 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a30f807f23d49ccadc653d246623d74 [term 1 LEADER]: Becoming Leader. State: Replica: 5a30f807f23d49ccadc653d246623d74, State: Running, Role: LEADER
I20260806 15:06:51.429601 27063 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a30f807f23d49ccadc653d246623d74 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:06:51.429644 27068 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a30f807f23d49ccadc653d246623d74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a30f807f23d49ccadc653d246623d74" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43877 } }
I20260806 15:06:51.429952 27068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a30f807f23d49ccadc653d246623d74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a30f807f23d49ccadc653d246623d74. Latest consensus state: current_term: 1 leader_uuid: "5a30f807f23d49ccadc653d246623d74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a30f807f23d49ccadc653d246623d74" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43877 } } }
I20260806 15:06:51.429968 27073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a30f807f23d49ccadc653d246623d74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a30f807f23d49ccadc653d246623d74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a30f807f23d49ccadc653d246623d74" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 43877 } } }
I20260806 15:06:51.430042 27068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a30f807f23d49ccadc653d246623d74 [sys.catalog]: This master's current role is: LEADER
I20260806 15:06:51.430071 27073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a30f807f23d49ccadc653d246623d74 [sys.catalog]: This master's current role is: LEADER
I20260806 15:06:51.430224 27082 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:06:51.430713 27082 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:06:51.432132 27082 catalog_manager.cc:1347] Generated new cluster ID: 9704726458374ceb90705b4619017e7c
I20260806 15:06:51.432185 27082 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:06:51.437228 27082 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:06:51.437927 27082 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:06:51.447592 27082 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a30f807f23d49ccadc653d246623d74: Generated new TSK 0
I20260806 15:06:51.447741 27082 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:06:51.449304 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:0
--local_ip_for_outbound_sockets=127.0.236.65
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:43877
--builtin_ntp_servers=127.0.236.84:43085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 15:06:51.529223 27100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:51.529467 27100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:51.529500 27100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:51.530752 27100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:51.530792 27100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:51.530817 27100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:06:51.532198 27100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:43085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:43877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:51.532426 27100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:51.532707 27100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:51.538275 27110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:51.538291 27109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:51.538501 27100 server_base.cc:1034] running on GCE node
W20260806 15:06:51.538291 27112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:51.538789 27100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:51.539024 27100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:51.540156 27100 hybrid_clock.cc:648] HybridClock initialized: now 1786028811540142 us; error 32 us; skew 500 ppm
I20260806 15:06:51.541286 27100 webserver.cc:460] Webserver started at http://127.0.236.65:38515/ using document root <none> and password file <none>
I20260806 15:06:51.541507 27100 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:51.541556 27100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:51.541680 27100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:06:51.542670 27100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-0/data/instance:
uuid: "b150b54f72604969be0fe2f49726e718"
format_stamp: "Formatted at 2026-08-06 15:06:51 on dist-test-slave-tc2s"
server_key: "0d3479ef559f07b8656e85fc05021615"
server_key_iv: "cd804350634961c9c065d55e0bb8460e"
server_key_version: "encryptionkey@0"
I20260806 15:06:51.542990 27100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-0/wal/instance:
uuid: "b150b54f72604969be0fe2f49726e718"
format_stamp: "Formatted at 2026-08-06 15:06:51 on dist-test-slave-tc2s"
server_key: "0d3479ef559f07b8656e85fc05021615"
server_key_iv: "cd804350634961c9c065d55e0bb8460e"
server_key_version: "encryptionkey@0"
I20260806 15:06:51.544199 27100 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 15:06:51.544929 27124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:51.545087 27100 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:06:51.545169 27100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "b150b54f72604969be0fe2f49726e718"
format_stamp: "Formatted at 2026-08-06 15:06:51 on dist-test-slave-tc2s"
server_key: "0d3479ef559f07b8656e85fc05021615"
server_key_iv: "cd804350634961c9c065d55e0bb8460e"
server_key_version: "encryptionkey@0"
I20260806 15:06:51.545236 27100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:51.594283 27100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:51.594573 27100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:51.594699 27100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:51.594913 27100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:06:51.595222 27100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:06:51.595256 27100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:51.595285 27100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:06:51.595304 27100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:51.601042 27100 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:44783
I20260806 15:06:51.601094 27286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:44783 every 8 connection(s)
I20260806 15:06:51.601377 27100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:06:51.605031 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 27100
I20260806 15:06:51.605131 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-0/wal/instance
I20260806 15:06:51.605311 945 external_mini_cluster.cc:1428] Setting key 271e53c57fb52d924f44afd62f283c3f
I20260806 15:06:51.606263 27066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.602003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:51355 (local address 127.0.236.126:43877)
0806 15:06:51.602061 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:51.602063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:51.602282 (+ 219us) server_negotiation.cc:407] Connection header received
0806 15:06:51.602731 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:51.602732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:51.602742 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:51.602804 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:51.603461 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.603931 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.604823 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.605085 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.605154 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.606020 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:51.606021 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:51.606022 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:06:51.606027 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:51.606044 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:06:51.606049 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:51.606098 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:51.606161 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 15:06:51.606185 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 15:06:51.606261 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:0
--local_ip_for_outbound_sockets=127.0.236.66
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:43877
--builtin_ntp_servers=127.0.236.84:43085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 15:06:51.606426 27295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.602004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43877 (local address 127.0.236.65:51355)
0806 15:06:51.602174 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:51.602206 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:51.602651 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:51.602887 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:51.602890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:51.602899 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:06:51.603396 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.603400 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.603996 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.603996 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.604763 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.604765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.605168 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.605170 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.605309 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.605848 (+ 539us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:06:51.605848 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:06:51.605886 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:06:51.605937 (+ 51us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:06:51.606109 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:06:51.606113 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:06:51.606115 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 15:06:51.606158 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 15:06:51.606234 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":88,"threads_started":1}
I20260806 15:06:51.608019 27287 heartbeater.cc:347] Connected to a master server at 127.0.236.126:43877
I20260806 15:06:51.608103 27287 heartbeater.cc:464] Registering TS with master...
I20260806 15:06:51.608268 27287 heartbeater.cc:511] Master 127.0.236.126:43877 requested a full tablet report, sending...
I20260806 15:06:51.608757 27014 ts_manager.cc:194] Registered new tserver with Master: b150b54f72604969be0fe2f49726e718 (127.0.236.65:44783)
I20260806 15:06:51.609371 27014 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.236.65:51355
W20260806 15:06:51.685459 27296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:51.685643 27296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:51.685671 27296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:51.686779 27296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:51.686817 27296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:51.686851 27296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:06:51.688138 27296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:43085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:43877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:51.688349 27296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:51.688601 27296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:51.693171 27308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:51.693181 27307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:51.693332 27310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:51.693367 27296 server_base.cc:1034] running on GCE node
I20260806 15:06:51.693589 27296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:51.693815 27296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:51.694957 27296 hybrid_clock.cc:648] HybridClock initialized: now 1786028811694954 us; error 42 us; skew 500 ppm
I20260806 15:06:51.696142 27296 webserver.cc:460] Webserver started at http://127.0.236.66:40125/ using document root <none> and password file <none>
I20260806 15:06:51.696344 27296 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:51.696390 27296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:51.696508 27296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:06:51.697466 27296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-1/data/instance:
uuid: "db29e6fbefae488cad39df459a277d18"
format_stamp: "Formatted at 2026-08-06 15:06:51 on dist-test-slave-tc2s"
server_key: "29861bdb64f017ad3b61722fc2d57dfe"
server_key_iv: "923f35e9be2d40d3bca6d451b56dde96"
server_key_version: "encryptionkey@0"
I20260806 15:06:51.697773 27296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-1/wal/instance:
uuid: "db29e6fbefae488cad39df459a277d18"
format_stamp: "Formatted at 2026-08-06 15:06:51 on dist-test-slave-tc2s"
server_key: "29861bdb64f017ad3b61722fc2d57dfe"
server_key_iv: "923f35e9be2d40d3bca6d451b56dde96"
server_key_version: "encryptionkey@0"
I20260806 15:06:51.698920 27296 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:06:51.699563 27321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:51.699710 27296 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:06:51.699790 27296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "db29e6fbefae488cad39df459a277d18"
format_stamp: "Formatted at 2026-08-06 15:06:51 on dist-test-slave-tc2s"
server_key: "29861bdb64f017ad3b61722fc2d57dfe"
server_key_iv: "923f35e9be2d40d3bca6d451b56dde96"
server_key_version: "encryptionkey@0"
I20260806 15:06:51.699849 27296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:51.714871 27296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:51.715103 27296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:51.715214 27296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:51.715394 27296 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:06:51.715672 27296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:06:51.715703 27296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:51.715729 27296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:06:51.715754 27296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:51.720922 27296 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:46487
I20260806 15:06:51.720957 27495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:46487 every 8 connection(s)
I20260806 15:06:51.721228 27296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
I20260806 15:06:51.721463 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 27296
I20260806 15:06:51.721556 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-1/wal/instance
I20260806 15:06:51.721748 945 external_mini_cluster.cc:1428] Setting key 03ac31f14eda3d87114b5805e8ff57d4
I20260806 15:06:51.722658 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:0
--local_ip_for_outbound_sockets=127.0.236.67
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:43877
--builtin_ntp_servers=127.0.236.84:43085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 15:06:51.726751 27066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.721938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:36165 (local address 127.0.236.126:43877)
0806 15:06:51.721992 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:51.721994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:51.722219 (+ 225us) server_negotiation.cc:407] Connection header received
0806 15:06:51.722698 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:51.722699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:51.722711 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:51.722775 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:51.723456 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.724113 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.725448 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.725687 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.725756 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.726487 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:51.726489 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:51.726489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:06:51.726494 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:51.726512 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:06:51.726517 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:51.726577 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:51.726643 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 15:06:51.726672 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 15:06:51.726931 27499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.721881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43877 (local address 127.0.236.66:36165)
0806 15:06:51.722099 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:51.722139 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:51.722608 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:51.722905 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:51.722908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:51.722916 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:06:51.723362 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.723365 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.724217 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.724218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.725367 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.725370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.725761 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.725761 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.725863 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.726355 (+ 492us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:06:51.726356 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:06:51.726381 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:06:51.726405 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:06:51.726581 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:06:51.726583 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:06:51.726585 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 15:06:51.726623 (+ 38us) client_negotiation.cc:240] Negotiation successful
0806 15:06:51.726695 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":92,"threads_started":1}
I20260806 15:06:51.727363 27496 heartbeater.cc:347] Connected to a master server at 127.0.236.126:43877
I20260806 15:06:51.727455 27496 heartbeater.cc:464] Registering TS with master...
I20260806 15:06:51.727615 27496 heartbeater.cc:511] Master 127.0.236.126:43877 requested a full tablet report, sending...
I20260806 15:06:51.727964 27014 ts_manager.cc:194] Registered new tserver with Master: db29e6fbefae488cad39df459a277d18 (127.0.236.66:46487)
I20260806 15:06:51.728430 27014 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.236.66:36165
W20260806 15:06:51.802699 27503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:51.802881 27503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:51.802901 27503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:51.804026 27503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:51.804066 27503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:51.804090 27503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:06:51.805349 27503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:43085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:43877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:51.805527 27503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:51.805750 27503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:51.810302 27509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:51.810271 27510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:51.810285 27503 server_base.cc:1034] running on GCE node
W20260806 15:06:51.810271 27512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:51.810696 27503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:51.810900 27503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:51.812037 27503 hybrid_clock.cc:648] HybridClock initialized: now 1786028811812024 us; error 31 us; skew 500 ppm
I20260806 15:06:51.813135 27503 webserver.cc:460] Webserver started at http://127.0.236.67:33255/ using document root <none> and password file <none>
I20260806 15:06:51.813350 27503 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:51.813393 27503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:51.813519 27503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:06:51.814522 27503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-2/data/instance:
uuid: "304ba8f580f1437fb16006b2a7b1d6e5"
format_stamp: "Formatted at 2026-08-06 15:06:51 on dist-test-slave-tc2s"
server_key: "a0369c9fb291ec227e883c0d52898df4"
server_key_iv: "1c1d74c0f22eb916789c01332d7abb41"
server_key_version: "encryptionkey@0"
I20260806 15:06:51.814828 27503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-2/wal/instance:
uuid: "304ba8f580f1437fb16006b2a7b1d6e5"
format_stamp: "Formatted at 2026-08-06 15:06:51 on dist-test-slave-tc2s"
server_key: "a0369c9fb291ec227e883c0d52898df4"
server_key_iv: "1c1d74c0f22eb916789c01332d7abb41"
server_key_version: "encryptionkey@0"
I20260806 15:06:51.816046 27503 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:06:51.816668 27519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:51.816792 27503 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:06:51.816866 27503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "304ba8f580f1437fb16006b2a7b1d6e5"
format_stamp: "Formatted at 2026-08-06 15:06:51 on dist-test-slave-tc2s"
server_key: "a0369c9fb291ec227e883c0d52898df4"
server_key_iv: "1c1d74c0f22eb916789c01332d7abb41"
server_key_version: "encryptionkey@0"
I20260806 15:06:51.816931 27503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:51.825935 27503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:51.826164 27503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:51.826265 27503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:51.826454 27503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:06:51.826726 27503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:06:51.826758 27503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:51.826778 27503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:06:51.826802 27503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:51.832129 27503 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:46727
I20260806 15:06:51.832195 27691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:46727 every 8 connection(s)
I20260806 15:06:51.832433 27503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
I20260806 15:06:51.836640 27066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.833037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:60641 (local address 127.0.236.126:43877)
0806 15:06:51.833079 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:51.833081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:51.833260 (+ 179us) server_negotiation.cc:407] Connection header received
0806 15:06:51.833669 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:51.833670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:51.833679 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:51.833726 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:51.834313 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.834792 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.835655 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.835827 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.835868 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.836458 (+ 590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:51.836460 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:51.836460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:06:51.836464 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:51.836478 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:06:51.836482 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:51.836519 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:51.836565 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 15:06:51.836585 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 15:06:51.836809 27696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.832998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:43877 (local address 127.0.236.67:60641)
0806 15:06:51.833166 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:51.833200 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:51.833598 (+ 398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:51.833774 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:51.833777 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:51.833785 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:06:51.834245 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.834249 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.834850 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.834851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.835597 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.835599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.835876 (+ 277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.835877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.835969 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.836338 (+ 369us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:06:51.836339 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:06:51.836362 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:06:51.836382 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:06:51.836522 (+ 140us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:06:51.836524 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:06:51.836526 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 15:06:51.836566 (+ 40us) client_negotiation.cc:240] Negotiation successful
0806 15:06:51.836634 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":84,"threads_started":1}
I20260806 15:06:51.837224 27692 heartbeater.cc:347] Connected to a master server at 127.0.236.126:43877
I20260806 15:06:51.837302 27692 heartbeater.cc:464] Registering TS with master...
I20260806 15:06:51.837438 27692 heartbeater.cc:511] Master 127.0.236.126:43877 requested a full tablet report, sending...
I20260806 15:06:51.837742 27014 ts_manager.cc:194] Registered new tserver with Master: 304ba8f580f1437fb16006b2a7b1d6e5 (127.0.236.67:46727)
I20260806 15:06:51.838104 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 27503
I20260806 15:06:51.838121 27014 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.236.67:60641
I20260806 15:06:51.838201 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-2/wal/instance
I20260806 15:06:51.838402 945 external_mini_cluster.cc:1428] Setting key 8a1cb6b598bbc60854a2162778a3a7de
I20260806 15:06:51.839160 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:06:51.842849 27066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.840363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55932 (local address 127.0.236.126:43877)
0806 15:06:51.840404 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:51.840406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:51.840506 (+ 100us) server_negotiation.cc:407] Connection header received
0806 15:06:51.840565 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:51.840566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:51.840575 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:51.840617 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:51.840753 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.841202 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.841919 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.842101 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.842151 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.842641 (+ 490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:51.842642 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:51.842643 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:06:51.842646 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:51.842659 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:06:51.842662 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:51.842706 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:51.842764 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 15:06:51.842783 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 15:06:51.844009 27014 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55932:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:06:51.845423 27014 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:06:51.851269 27705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.847217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:46727 (local address 127.0.0.1:51860)
0806 15:06:51.847305 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:51.847340 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:51.847397 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:51.848053 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:51.848054 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:51.848055 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:51.848089 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.848090 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.849081 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.849082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.850374 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.850376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.850981 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.850982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.851065 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.851079 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 15:06:51.851140 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 15:06:51.851193 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":32,"thread_start_us":46,"threads_started":1}
I20260806 15:06:51.851368 27711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.847583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44088 (local address 127.0.236.66:46487)
0806 15:06:51.847791 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:51.847795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:51.847890 (+ 95us) server_negotiation.cc:407] Connection header received
0806 15:06:51.847951 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:51.847953 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:51.847963 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:51.848104 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:51.848286 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.849310 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.850329 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.851062 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.851136 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.851225 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:51.851251 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 15:06:51.851287 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":128,"threads_started":1}
I20260806 15:06:51.851488 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.847394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:46487 (local address 127.0.0.1:44088)
0806 15:06:51.847820 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:51.847829 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:51.847885 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:51.848170 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:51.848171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:51.848174 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:51.848214 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.848216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.849383 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.849384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.850260 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.850262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.851133 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.851134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.851189 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.851192 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:06:51.851229 (+ 37us) client_negotiation.cc:240] Negotiation successful
0806 15:06:51.851260 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":64,"threads_started":1}
I20260806 15:06:51.851660 27708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.847337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51860 (local address 127.0.236.67:46727)
0806 15:06:51.847697 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:51.847701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:51.847711 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:06:51.847937 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:51.847943 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:51.847960 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:51.848155 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:51.848235 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.849005 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.850453 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.850897 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.850965 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.851051 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:51.851553 (+ 502us) server_negotiation.cc:299] Negotiation successful
0806 15:06:51.851590 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":84,"threads_started":1}
I20260806 15:06:51.851794 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.847509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54334 (local address 127.0.236.65:44783)
0806 15:06:51.847672 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:51.847676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:51.847684 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:06:51.847717 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:51.847718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:51.847734 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:51.847815 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:51.848011 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.849112 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.850443 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.851120 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.851175 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.851255 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:51.851670 (+ 415us) server_negotiation.cc:299] Negotiation successful
0806 15:06:51.851714 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260806 15:06:51.851876 27706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.847235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:44783 (local address 127.0.0.1:54334)
0806 15:06:51.847381 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:51.847390 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:51.847438 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:51.847858 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:51.847861 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:51.847863 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:51.847938 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.847939 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.849189 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.849190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.850346 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.850349 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.851290 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.851291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.851530 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.851536 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 15:06:51.851774 (+ 238us) client_negotiation.cc:240] Negotiation successful
0806 15:06:51.851816 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":42,"thread_start_us":86,"threads_started":1}
I20260806 15:06:51.852205 27393 tablet_service.cc:1467] Processing CreateTablet for tablet ea5fb3d2775b4b4586d752c557213abc (DEFAULT_TABLE table=test-table [id=ccfdc6edf581481ea7a285c35f50f53a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:06:51.852388 27587 tablet_service.cc:1467] Processing CreateTablet for tablet ea5fb3d2775b4b4586d752c557213abc (DEFAULT_TABLE table=test-table [id=ccfdc6edf581481ea7a285c35f50f53a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:06:51.852555 27393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea5fb3d2775b4b4586d752c557213abc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:51.852713 27587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea5fb3d2775b4b4586d752c557213abc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:51.854518 27193 tablet_service.cc:1467] Processing CreateTablet for tablet ea5fb3d2775b4b4586d752c557213abc (DEFAULT_TABLE table=test-table [id=ccfdc6edf581481ea7a285c35f50f53a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:06:51.854838 27193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea5fb3d2775b4b4586d752c557213abc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:51.855063 27712 tablet_bootstrap.cc:492] T ea5fb3d2775b4b4586d752c557213abc P db29e6fbefae488cad39df459a277d18: Bootstrap starting.
I20260806 15:06:51.855350 27713 tablet_bootstrap.cc:492] T ea5fb3d2775b4b4586d752c557213abc P 304ba8f580f1437fb16006b2a7b1d6e5: Bootstrap starting.
I20260806 15:06:51.855683 27712 tablet_bootstrap.cc:654] T ea5fb3d2775b4b4586d752c557213abc P db29e6fbefae488cad39df459a277d18: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:51.855944 27712 log.cc:824] T ea5fb3d2775b4b4586d752c557213abc P db29e6fbefae488cad39df459a277d18: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:51.855954 27713 tablet_bootstrap.cc:654] T ea5fb3d2775b4b4586d752c557213abc P 304ba8f580f1437fb16006b2a7b1d6e5: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:51.856246 27713 log.cc:824] T ea5fb3d2775b4b4586d752c557213abc P 304ba8f580f1437fb16006b2a7b1d6e5: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:51.856611 27712 tablet_bootstrap.cc:492] T ea5fb3d2775b4b4586d752c557213abc P db29e6fbefae488cad39df459a277d18: No bootstrap required, opened a new log
I20260806 15:06:51.856683 27712 ts_tablet_manager.cc:1397] T ea5fb3d2775b4b4586d752c557213abc P db29e6fbefae488cad39df459a277d18: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:06:51.857007 27713 tablet_bootstrap.cc:492] T ea5fb3d2775b4b4586d752c557213abc P 304ba8f580f1437fb16006b2a7b1d6e5: No bootstrap required, opened a new log
I20260806 15:06:51.857106 27713 ts_tablet_manager.cc:1397] T ea5fb3d2775b4b4586d752c557213abc P 304ba8f580f1437fb16006b2a7b1d6e5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:06:51.858008 27721 tablet_bootstrap.cc:492] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718: Bootstrap starting.
I20260806 15:06:51.858615 27721 tablet_bootstrap.cc:654] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:51.858927 27721 log.cc:824] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:51.859000 27712 raft_consensus.cc:348] T ea5fb3d2775b4b4586d752c557213abc P db29e6fbefae488cad39df459a277d18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "304ba8f580f1437fb16006b2a7b1d6e5" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46727 } } peers { permanent_uuid: "db29e6fbefae488cad39df459a277d18" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 46487 } } peers { permanent_uuid: "b150b54f72604969be0fe2f49726e718" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 44783 } }
I20260806 15:06:51.859153 27712 raft_consensus.cc:374] T ea5fb3d2775b4b4586d752c557213abc P db29e6fbefae488cad39df459a277d18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:51.859194 27712 raft_consensus.cc:729] T ea5fb3d2775b4b4586d752c557213abc P db29e6fbefae488cad39df459a277d18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db29e6fbefae488cad39df459a277d18, State: Initialized, Role: FOLLOWER
I20260806 15:06:51.859313 27712 consensus_queue.cc:260] T ea5fb3d2775b4b4586d752c557213abc P db29e6fbefae488cad39df459a277d18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "304ba8f580f1437fb16006b2a7b1d6e5" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46727 } } peers { permanent_uuid: "db29e6fbefae488cad39df459a277d18" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 46487 } } peers { permanent_uuid: "b150b54f72604969be0fe2f49726e718" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 44783 } }
I20260806 15:06:51.859283 27713 raft_consensus.cc:348] T ea5fb3d2775b4b4586d752c557213abc P 304ba8f580f1437fb16006b2a7b1d6e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "304ba8f580f1437fb16006b2a7b1d6e5" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46727 } } peers { permanent_uuid: "db29e6fbefae488cad39df459a277d18" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 46487 } } peers { permanent_uuid: "b150b54f72604969be0fe2f49726e718" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 44783 } }
I20260806 15:06:51.859431 27713 raft_consensus.cc:374] T ea5fb3d2775b4b4586d752c557213abc P 304ba8f580f1437fb16006b2a7b1d6e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:51.859472 27713 raft_consensus.cc:729] T ea5fb3d2775b4b4586d752c557213abc P 304ba8f580f1437fb16006b2a7b1d6e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 304ba8f580f1437fb16006b2a7b1d6e5, State: Initialized, Role: FOLLOWER
I20260806 15:06:51.859557 27712 ts_tablet_manager.cc:1428] T ea5fb3d2775b4b4586d752c557213abc P db29e6fbefae488cad39df459a277d18: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 15:06:51.859606 27496 heartbeater.cc:503] Master 127.0.236.126:43877 was elected leader, sending a full tablet report...
I20260806 15:06:51.859606 27713 consensus_queue.cc:260] T ea5fb3d2775b4b4586d752c557213abc P 304ba8f580f1437fb16006b2a7b1d6e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "304ba8f580f1437fb16006b2a7b1d6e5" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46727 } } peers { permanent_uuid: "db29e6fbefae488cad39df459a277d18" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 46487 } } peers { permanent_uuid: "b150b54f72604969be0fe2f49726e718" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 44783 } }
I20260806 15:06:51.859849 27713 ts_tablet_manager.cc:1428] T ea5fb3d2775b4b4586d752c557213abc P 304ba8f580f1437fb16006b2a7b1d6e5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 15:06:51.859880 27692 heartbeater.cc:503] Master 127.0.236.126:43877 was elected leader, sending a full tablet report...
I20260806 15:06:51.860200 27721 tablet_bootstrap.cc:492] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718: No bootstrap required, opened a new log
I20260806 15:06:51.860267 27721 ts_tablet_manager.cc:1397] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 15:06:51.862138 27721 raft_consensus.cc:348] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "304ba8f580f1437fb16006b2a7b1d6e5" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46727 } } peers { permanent_uuid: "db29e6fbefae488cad39df459a277d18" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 46487 } } peers { permanent_uuid: "b150b54f72604969be0fe2f49726e718" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 44783 } }
I20260806 15:06:51.862252 27721 raft_consensus.cc:374] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:51.862282 27721 raft_consensus.cc:729] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b150b54f72604969be0fe2f49726e718, State: Initialized, Role: FOLLOWER
I20260806 15:06:51.862382 27721 consensus_queue.cc:260] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "304ba8f580f1437fb16006b2a7b1d6e5" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46727 } } peers { permanent_uuid: "db29e6fbefae488cad39df459a277d18" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 46487 } } peers { permanent_uuid: "b150b54f72604969be0fe2f49726e718" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 44783 } }
I20260806 15:06:51.862607 27721 ts_tablet_manager.cc:1428] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:06:51.862664 27287 heartbeater.cc:503] Master 127.0.236.126:43877 was elected leader, sending a full tablet report...
I20260806 15:06:51.889622 27729 raft_consensus.cc:482] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:06:51.889694 27729 raft_consensus.cc:504] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "304ba8f580f1437fb16006b2a7b1d6e5" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46727 } } peers { permanent_uuid: "db29e6fbefae488cad39df459a277d18" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 46487 } } peers { permanent_uuid: "b150b54f72604969be0fe2f49726e718" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 44783 } }
I20260806 15:06:51.889966 27729 leader_election.cc:288] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 304ba8f580f1437fb16006b2a7b1d6e5 (127.0.236.67:46727), db29e6fbefae488cad39df459a277d18 (127.0.236.66:46487)
I20260806 15:06:51.892516 27295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.890098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:46727 (local address 127.0.236.65:53069)
0806 15:06:51.890142 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:51.890152 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:51.890188 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:51.890329 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:51.890330 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:51.890331 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:51.890384 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.890386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.891034 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.891035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.891771 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.891773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.892311 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.892312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.892374 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.892378 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:06:51.892426 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 15:06:51.892467 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 15:06:51.892524 27708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.890176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:53069 (local address 127.0.236.67:46727)
0806 15:06:51.890229 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:51.890230 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:51.890239 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:06:51.890260 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:51.890261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:51.890268 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:51.890324 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:51.890458 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.890962 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.891864 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.892236 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.892285 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.892291 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:51.892435 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 15:06:51.892470 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 15:06:51.892657 27730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.890191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:46487 (local address 127.0.236.65:56535)
0806 15:06:51.890330 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:51.890340 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:51.890391 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:51.890511 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:51.890511 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:51.890513 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:51.890576 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.890578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.891151 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.891152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.891904 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:51.891907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.892405 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.892406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:51.892497 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.892501 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:06:51.892556 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 15:06:51.892590 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":1,"thread_start_us":77,"threads_started":1}
I20260806 15:06:51.892777 27711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.890254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:56535 (local address 127.0.236.66:46487)
0806 15:06:51.890296 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:51.890298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:51.890395 (+ 97us) server_negotiation.cc:407] Connection header received
0806 15:06:51.890443 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:51.890443 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:51.890451 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:51.890495 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:51.890634 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.891069 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.891981 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.892328 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.892393 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.892399 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:51.892682 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 15:06:51.892720 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 15:06:51.892799 27623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea5fb3d2775b4b4586d752c557213abc" candidate_uuid: "b150b54f72604969be0fe2f49726e718" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "304ba8f580f1437fb16006b2a7b1d6e5" is_pre_election: true
I20260806 15:06:51.892904 27623 raft_consensus.cc:2454] T ea5fb3d2775b4b4586d752c557213abc P 304ba8f580f1437fb16006b2a7b1d6e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b150b54f72604969be0fe2f49726e718 in term 0.
I20260806 15:06:51.893016 27424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea5fb3d2775b4b4586d752c557213abc" candidate_uuid: "b150b54f72604969be0fe2f49726e718" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db29e6fbefae488cad39df459a277d18" is_pre_election: true
I20260806 15:06:51.893113 27125 leader_election.cc:302] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 304ba8f580f1437fb16006b2a7b1d6e5, b150b54f72604969be0fe2f49726e718; no voters:
I20260806 15:06:51.893132 27424 raft_consensus.cc:2454] T ea5fb3d2775b4b4586d752c557213abc P db29e6fbefae488cad39df459a277d18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b150b54f72604969be0fe2f49726e718 in term 0.
I20260806 15:06:51.893214 27729 raft_consensus.cc:2781] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:06:51.893267 27729 raft_consensus.cc:482] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:06:51.893290 27729 raft_consensus.cc:3037] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:51.893841 27729 raft_consensus.cc:504] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "304ba8f580f1437fb16006b2a7b1d6e5" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46727 } } peers { permanent_uuid: "db29e6fbefae488cad39df459a277d18" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 46487 } } peers { permanent_uuid: "b150b54f72604969be0fe2f49726e718" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 44783 } }
I20260806 15:06:51.893967 27729 leader_election.cc:288] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718 [CANDIDATE]: Term 1 election: Requested vote from peers 304ba8f580f1437fb16006b2a7b1d6e5 (127.0.236.67:46727), db29e6fbefae488cad39df459a277d18 (127.0.236.66:46487)
I20260806 15:06:51.894089 27623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea5fb3d2775b4b4586d752c557213abc" candidate_uuid: "b150b54f72604969be0fe2f49726e718" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "304ba8f580f1437fb16006b2a7b1d6e5"
I20260806 15:06:51.894131 27424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea5fb3d2775b4b4586d752c557213abc" candidate_uuid: "b150b54f72604969be0fe2f49726e718" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db29e6fbefae488cad39df459a277d18"
I20260806 15:06:51.894153 27623 raft_consensus.cc:3037] T ea5fb3d2775b4b4586d752c557213abc P 304ba8f580f1437fb16006b2a7b1d6e5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:51.894204 27424 raft_consensus.cc:3037] T ea5fb3d2775b4b4586d752c557213abc P db29e6fbefae488cad39df459a277d18 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:51.894774 27623 raft_consensus.cc:2454] T ea5fb3d2775b4b4586d752c557213abc P 304ba8f580f1437fb16006b2a7b1d6e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b150b54f72604969be0fe2f49726e718 in term 1.
I20260806 15:06:51.894917 27125 leader_election.cc:302] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 304ba8f580f1437fb16006b2a7b1d6e5, b150b54f72604969be0fe2f49726e718; no voters:
I20260806 15:06:51.895004 27424 raft_consensus.cc:2454] T ea5fb3d2775b4b4586d752c557213abc P db29e6fbefae488cad39df459a277d18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b150b54f72604969be0fe2f49726e718 in term 1.
I20260806 15:06:51.895022 27729 raft_consensus.cc:2781] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:06:51.895161 27729 raft_consensus.cc:686] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718 [term 1 LEADER]: Becoming Leader. State: Replica: b150b54f72604969be0fe2f49726e718, State: Running, Role: LEADER
I20260806 15:06:51.895246 27729 consensus_queue.cc:237] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "304ba8f580f1437fb16006b2a7b1d6e5" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46727 } } peers { permanent_uuid: "db29e6fbefae488cad39df459a277d18" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 46487 } } peers { permanent_uuid: "b150b54f72604969be0fe2f49726e718" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 44783 } }
I20260806 15:06:51.895888 27014 catalog_manager.cc:5495] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718 reported cstate change: term changed from 0 to 1, leader changed from <none> to b150b54f72604969be0fe2f49726e718 (127.0.236.65). New cstate: current_term: 1 leader_uuid: "b150b54f72604969be0fe2f49726e718" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "304ba8f580f1437fb16006b2a7b1d6e5" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db29e6fbefae488cad39df459a277d18" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 46487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b150b54f72604969be0fe2f49726e718" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 44783 } health_report { overall_health: HEALTHY } } }
I20260806 15:06:51.910245 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.908136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54336 (local address 127.0.236.65:44783)
0806 15:06:51.908173 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:51.908175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:51.908196 (+ 21us) server_negotiation.cc:407] Connection header received
0806 15:06:51.908262 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:51.908263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:51.908280 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:51.908335 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:06:51.908446 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.908865 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.909566 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:51.909746 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:51.909796 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:51.909955 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:06:51.910043 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:06:51.910109 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:51.910146 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 15:06:51.910163 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 15:06:51.911649 27424 raft_consensus.cc:1261] T ea5fb3d2775b4b4586d752c557213abc P db29e6fbefae488cad39df459a277d18 [term 1 FOLLOWER]: Refusing update from remote peer b150b54f72604969be0fe2f49726e718: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:06:51.911660 27623 raft_consensus.cc:1261] T ea5fb3d2775b4b4586d752c557213abc P 304ba8f580f1437fb16006b2a7b1d6e5 [term 1 FOLLOWER]: Refusing update from remote peer b150b54f72604969be0fe2f49726e718: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:06:51.911911 27729 consensus_queue.cc:1035] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db29e6fbefae488cad39df459a277d18" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 46487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:06:51.912040 27731 consensus_queue.cc:1035] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718 [LEADER]: Connected to new peer: Peer: permanent_uuid: "304ba8f580f1437fb16006b2a7b1d6e5" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 46727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:06:51.914816 27735 mvcc.cc:205] Tried to move back new op lower bound from 7315574013587992576 to 7315574013523398656. Current Snapshot: MvccSnapshot[applied={T|T < 7315574013587992576 or (T in {7315574013587992576})}]
I20260806 15:06:51.915041 27734 mvcc.cc:205] Tried to move back new op lower bound from 7315574013587992576 to 7315574013523398656. Current Snapshot: MvccSnapshot[applied={T|T < 7315574013587992576}]
I20260806 15:06:51.915160 27736 mvcc.cc:205] Tried to move back new op lower bound from 7315574013587992576 to 7315574013523398656. Current Snapshot: MvccSnapshot[applied={T|T < 7315574013587992576 or (T in {7315574013587992576})}]
I20260806 15:06:51.916923 27013 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:06:51.916998 27013 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:06:51.917629 27013 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5a30f807f23d49ccadc653d246623d74: Sending DeleteTablet for 3 replicas of tablet ea5fb3d2775b4b4586d752c557213abc
I20260806 15:06:51.918000 27393 tablet_service.cc:1514] Processing DeleteTablet for tablet ea5fb3d2775b4b4586d752c557213abc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:06:51 UTC) from {username='slave'} at 127.0.0.1:44088
I20260806 15:06:51.918000 27193 tablet_service.cc:1514] Processing DeleteTablet for tablet ea5fb3d2775b4b4586d752c557213abc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:06:51 UTC) from {username='slave'} at 127.0.0.1:54334
I20260806 15:06:51.918013 27587 tablet_service.cc:1514] Processing DeleteTablet for tablet ea5fb3d2775b4b4586d752c557213abc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:06:51 UTC) from {username='slave'} at 127.0.0.1:51860
I20260806 15:06:51.918304 27747 tablet_replica.cc:330] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718: stopping tablet replica
I20260806 15:06:51.918401 27747 raft_consensus.cc:2229] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718 [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:06:51.918426 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 27100
I20260806 15:06:51.918522 27747 raft_consensus.cc:2258] T ea5fb3d2775b4b4586d752c557213abc P b150b54f72604969be0fe2f49726e718 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:06:51.926005 27748 tablet_replica.cc:330] T ea5fb3d2775b4b4586d752c557213abc P db29e6fbefae488cad39df459a277d18: stopping tablet replica
I20260806 15:06:51.926092 27748 raft_consensus.cc:2229] T ea5fb3d2775b4b4586d752c557213abc P db29e6fbefae488cad39df459a277d18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:06:51.926190 27748 raft_consensus.cc:2258] T ea5fb3d2775b4b4586d752c557213abc P db29e6fbefae488cad39df459a277d18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:06:51.926201 27749 tablet_replica.cc:330] T ea5fb3d2775b4b4586d752c557213abc P 304ba8f580f1437fb16006b2a7b1d6e5: stopping tablet replica
I20260806 15:06:51.926326 27749 raft_consensus.cc:2229] T ea5fb3d2775b4b4586d752c557213abc P 304ba8f580f1437fb16006b2a7b1d6e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:06:51.926419 27749 raft_consensus.cc:2258] T ea5fb3d2775b4b4586d752c557213abc P 304ba8f580f1437fb16006b2a7b1d6e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:06:51.926721 27748 ts_tablet_manager.cc:1905] T ea5fb3d2775b4b4586d752c557213abc P db29e6fbefae488cad39df459a277d18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:06:51.926800 27749 ts_tablet_manager.cc:1905] T ea5fb3d2775b4b4586d752c557213abc P 304ba8f580f1437fb16006b2a7b1d6e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:06:51.928149 27748 ts_tablet_manager.cc:1918] T ea5fb3d2775b4b4586d752c557213abc P db29e6fbefae488cad39df459a277d18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:06:51.928213 27748 log.cc:1196] T ea5fb3d2775b4b4586d752c557213abc P db29e6fbefae488cad39df459a277d18: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-1/wal/wals/ea5fb3d2775b4b4586d752c557213abc
I20260806 15:06:51.928206 27749 ts_tablet_manager.cc:1918] T ea5fb3d2775b4b4586d752c557213abc P 304ba8f580f1437fb16006b2a7b1d6e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:06:51.928264 27749 log.cc:1196] T ea5fb3d2775b4b4586d752c557213abc P 304ba8f580f1437fb16006b2a7b1d6e5: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/ea5fb3d2775b4b4586d752c557213abc
I20260806 15:06:51.928452 27748 ts_tablet_manager.cc:1939] T ea5fb3d2775b4b4586d752c557213abc P db29e6fbefae488cad39df459a277d18: Deleting consensus metadata
I20260806 15:06:51.928495 27749 ts_tablet_manager.cc:1939] T ea5fb3d2775b4b4586d752c557213abc P 304ba8f580f1437fb16006b2a7b1d6e5: Deleting consensus metadata
I20260806 15:06:51.928874 26986 catalog_manager.cc:4841] TS 304ba8f580f1437fb16006b2a7b1d6e5 (127.0.236.67:46727): tablet ea5fb3d2775b4b4586d752c557213abc (table test-table [id=ccfdc6edf581481ea7a285c35f50f53a]) successfully deleted
I20260806 15:06:51.929013 26986 catalog_manager.cc:4841] TS db29e6fbefae488cad39df459a277d18 (127.0.236.66:46487): tablet ea5fb3d2775b4b4586d752c557213abc (table test-table [id=ccfdc6edf581481ea7a285c35f50f53a]) successfully deleted
W20260806 15:06:51.929376 26987 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.236.65:44783 (error 108)
I20260806 15:06:51.929615 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 27296
I20260806 15:06:51.929790 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:51.929670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:44783 (local address 127.0.0.1:54344)
0806 15:06:51.929730 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:51.929764 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.65:44783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 15:06:51.932798 26987 catalog_manager.cc:4584] TS b150b54f72604969be0fe2f49726e718 (127.0.236.65:44783): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ea5fb3d2775b4b4586d752c557213abc: Network error: Client connection negotiation failed: client connection to 127.0.236.65:44783: connect: Connection refused (error 111)
I20260806 15:06:51.936612 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 27503
I20260806 15:06:51.945712 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 26966
2026-08-06T15:06:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (668 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:06:51 dist-test-slave-tc2s krb5kdc[27756](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:06:51 dist-test-slave-tc2s krb5kdc[27756](info): set up 2 sockets
Aug 06 15:06:51 dist-test-slave-tc2s krb5kdc[27756](info): commencing operation
krb5kdc: starting...
W20260806 15:06:53.046902 945 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:06:53 dist-test-slave-tc2s krb5kdc[27756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:06:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:06:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:06:53.153721 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:44759
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:32789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:44759
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:06:53.231972 27772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:53.232160 27772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:53.232189 27772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:53.233325 27772 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:06:53.233367 27772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:53.233386 27772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:53.233402 27772 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:06:53.233433 27772 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:06:53.234895 27772 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:32789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:44759
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:44759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:53.235117 27772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:53.235307 27772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:53.239702 27778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:53.239730 27781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:53.239744 27784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:53.239804 27772 server_base.cc:1034] running on GCE node
I20260806 15:06:53.240097 27772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:53.240386 27772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:53.241523 27772 hybrid_clock.cc:648] HybridClock initialized: now 1786028813241511 us; error 30 us; skew 500 ppm
Aug 06 15:06:53 dist-test-slave-tc2s krb5kdc[27756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028813, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:06:53.246744 27772 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:06:53.247125 27772 webserver.cc:460] Webserver started at http://127.0.236.126:46261/ using document root <none> and password file <none>
I20260806 15:06:53.247309 27772 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:53.247356 27772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:53.247484 27772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:06:53.248416 27772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "5c466d15aa7149cdb4b113926549ac0c"
format_stamp: "Formatted at 2026-08-06 15:06:53 on dist-test-slave-tc2s"
server_key: "8fedca04e69c64f1b461d718f571e8d3"
server_key_iv: "8a55d262995bd2552d77ef8a56af5438"
server_key_version: "encryptionkey@0"
I20260806 15:06:53.248740 27772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "5c466d15aa7149cdb4b113926549ac0c"
format_stamp: "Formatted at 2026-08-06 15:06:53 on dist-test-slave-tc2s"
server_key: "8fedca04e69c64f1b461d718f571e8d3"
server_key_iv: "8a55d262995bd2552d77ef8a56af5438"
server_key_version: "encryptionkey@0"
I20260806 15:06:53.250006 27772 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 15:06:53.250660 27795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:53.250802 27772 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:06:53.250883 27772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "5c466d15aa7149cdb4b113926549ac0c"
format_stamp: "Formatted at 2026-08-06 15:06:53 on dist-test-slave-tc2s"
server_key: "8fedca04e69c64f1b461d718f571e8d3"
server_key_iv: "8a55d262995bd2552d77ef8a56af5438"
server_key_version: "encryptionkey@0"
I20260806 15:06:53.250950 27772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:53.257525 27772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:53.259495 27772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:53.259621 27772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:53.263330 27772 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:44759
I20260806 15:06:53.263360 27867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:44759 every 8 connection(s)
I20260806 15:06:53.263649 27772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:06:53.264393 27869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:53.266499 27869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c466d15aa7149cdb4b113926549ac0c: Bootstrap starting.
I20260806 15:06:53.267043 27869 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c466d15aa7149cdb4b113926549ac0c: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:53.267309 27869 log.cc:824] T 00000000000000000000000000000000 P 5c466d15aa7149cdb4b113926549ac0c: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:53.267457 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 27772
I20260806 15:06:53.267575 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:06:53.267755 945 external_mini_cluster.cc:1428] Setting key a5c7e02eccb64edb9e4bfd32df5bc2f9
I20260806 15:06:53.268057 27869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c466d15aa7149cdb4b113926549ac0c: No bootstrap required, opened a new log
I20260806 15:06:53.270612 27869 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c466d15aa7149cdb4b113926549ac0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c466d15aa7149cdb4b113926549ac0c" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44759 } }
I20260806 15:06:53.270774 27869 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c466d15aa7149cdb4b113926549ac0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:53.270818 27869 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c466d15aa7149cdb4b113926549ac0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c466d15aa7149cdb4b113926549ac0c, State: Initialized, Role: FOLLOWER
I20260806 15:06:53.270944 27869 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c466d15aa7149cdb4b113926549ac0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c466d15aa7149cdb4b113926549ac0c" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44759 } }
I20260806 15:06:53.271023 27869 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c466d15aa7149cdb4b113926549ac0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:06:53.271055 27869 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c466d15aa7149cdb4b113926549ac0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:06:53.271097 27869 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c466d15aa7149cdb4b113926549ac0c [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:53.271689 27869 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c466d15aa7149cdb4b113926549ac0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c466d15aa7149cdb4b113926549ac0c" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44759 } }
I20260806 15:06:53.271785 27869 leader_election.cc:302] T 00000000000000000000000000000000 P 5c466d15aa7149cdb4b113926549ac0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c466d15aa7149cdb4b113926549ac0c; no voters:
I20260806 15:06:53.271944 27869 leader_election.cc:288] T 00000000000000000000000000000000 P 5c466d15aa7149cdb4b113926549ac0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:06:53.271981 27877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c466d15aa7149cdb4b113926549ac0c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:06:53.272138 27877 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c466d15aa7149cdb4b113926549ac0c [term 1 LEADER]: Becoming Leader. State: Replica: 5c466d15aa7149cdb4b113926549ac0c, State: Running, Role: LEADER
I20260806 15:06:53.272190 27869 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c466d15aa7149cdb4b113926549ac0c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:06:53.272841 27877 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c466d15aa7149cdb4b113926549ac0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c466d15aa7149cdb4b113926549ac0c" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44759 } }
Aug 06 15:06:53 dist-test-slave-tc2s krb5kdc[27756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:06:53.274516 27878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c466d15aa7149cdb4b113926549ac0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c466d15aa7149cdb4b113926549ac0c. Latest consensus state: current_term: 1 leader_uuid: "5c466d15aa7149cdb4b113926549ac0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c466d15aa7149cdb4b113926549ac0c" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44759 } } }
I20260806 15:06:53.274606 27878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c466d15aa7149cdb4b113926549ac0c [sys.catalog]: This master's current role is: LEADER
I20260806 15:06:53.274999 27900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:06:53.275347 27883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c466d15aa7149cdb4b113926549ac0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c466d15aa7149cdb4b113926549ac0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c466d15aa7149cdb4b113926549ac0c" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44759 } } }
I20260806 15:06:53.275492 27883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c466d15aa7149cdb4b113926549ac0c [sys.catalog]: This master's current role is: LEADER
I20260806 15:06:53.275801 27900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:06:53.277086 27900 catalog_manager.cc:1347] Generated new cluster ID: f986c3508f264dcfa7d3df12a4aae61c
I20260806 15:06:53.277132 27900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:06:53.278304 27874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:53.268182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51304 (local address 127.0.236.126:44759)
0806 15:06:53.268336 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:53.268345 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:53.268369 (+ 24us) server_negotiation.cc:407] Connection header received
0806 15:06:53.268747 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:53.268753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:53.268794 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:53.268908 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:53.269646 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.270582 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.271608 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.272101 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.272484 (+ 383us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:53.274573 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:53.274577 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:53.274578 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:06:53.274589 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:53.276350 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:53.276717 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:53.276720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:53.276720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:53.276821 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:53.277049 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:53.277049 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:53.277050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:53.277330 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:53.277472 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:53.277981 (+ 509us) server_negotiation.cc:299] Negotiation successful
0806 15:06:53.278114 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
I20260806 15:06:53.288527 27900 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:06:53.289112 27900 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:06:53.298998 27900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c466d15aa7149cdb4b113926549ac0c: Generated new TSK 0
I20260806 15:06:53.299168 27900 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.236.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
I20260806 15:06:53.322214 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:0
--local_ip_for_outbound_sockets=127.0.236.65
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44759
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:32789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:06:53.401211 27913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:53.401402 27913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:53.401430 27913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:53.402557 27913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:53.402596 27913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:53.402623 27913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:06:53.404402 27913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:32789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:53.404645 27913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:53.404889 27913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:53.410629 27921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:53.410835 27920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:53.410629 27923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:53.411202 27913 server_base.cc:1034] running on GCE node
I20260806 15:06:53.411425 27913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:53.411672 27913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:53.412817 27913 hybrid_clock.cc:648] HybridClock initialized: now 1786028813412794 us; error 39 us; skew 500 ppm
Aug 06 15:06:53 dist-test-slave-tc2s krb5kdc[27756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028813, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:06:53.415131 27913 init.cc:377] Logged in from keytab as kudu/127.0.236.65@KRBTEST.COM (short username kudu)
I20260806 15:06:53.415530 27913 webserver.cc:460] Webserver started at http://127.0.236.65:32769/ using document root <none> and password file <none>
I20260806 15:06:53.415714 27913 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:53.415762 27913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:53.415890 27913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:06:53.416831 27913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-0/data/instance:
uuid: "3f7bb8d23aea4a2e88ada823fef5b286"
format_stamp: "Formatted at 2026-08-06 15:06:53 on dist-test-slave-tc2s"
server_key: "d63b01aef77beab179a511f360c24f1d"
server_key_iv: "710e42d23cb702242a2cd44d31e5ae82"
server_key_version: "encryptionkey@0"
I20260806 15:06:53.417111 27913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-0/wal/instance:
uuid: "3f7bb8d23aea4a2e88ada823fef5b286"
format_stamp: "Formatted at 2026-08-06 15:06:53 on dist-test-slave-tc2s"
server_key: "d63b01aef77beab179a511f360c24f1d"
server_key_iv: "710e42d23cb702242a2cd44d31e5ae82"
server_key_version: "encryptionkey@0"
I20260806 15:06:53.418295 27913 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:06:53.418910 27933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:53.419062 27913 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:06:53.419138 27913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "3f7bb8d23aea4a2e88ada823fef5b286"
format_stamp: "Formatted at 2026-08-06 15:06:53 on dist-test-slave-tc2s"
server_key: "d63b01aef77beab179a511f360c24f1d"
server_key_iv: "710e42d23cb702242a2cd44d31e5ae82"
server_key_version: "encryptionkey@0"
I20260806 15:06:53.419189 27913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:53.427793 27913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:53.429672 27913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:53.429775 27913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:53.429992 27913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:06:53.430241 27913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:06:53.430270 27913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:53.430295 27913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:06:53.430311 27913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:53.435612 27913 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:42421
I20260806 15:06:53.435678 28107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:42421 every 8 connection(s)
I20260806 15:06:53.435925 27913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:06:53.436188 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 27913
I20260806 15:06:53.436312 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-0/wal/instance
I20260806 15:06:53.436512 945 external_mini_cluster.cc:1428] Setting key fc112b84dd51c09b538f3bd94ae86537
Aug 06 15:06:53 dist-test-slave-tc2s krb5kdc[27756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028813, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.236.66@KRBTEST.COM; defaulting to no policy
I20260806 15:06:53.445676 27874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:53.436784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:38923 (local address 127.0.236.126:44759)
0806 15:06:53.437021 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:53.437022 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:53.437031 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:06:53.438056 (+ 1025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:53.438057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:53.438067 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:53.438106 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:53.438832 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.439571 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.440732 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.440930 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.440982 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:53.443125 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:53.443127 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:53.443127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:06:53.443135 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:53.444312 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:53.444759 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:53.444761 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:53.444761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:53.444819 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:53.445042 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:53.445043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:53.445044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:53.445156 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:53.445213 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:53.445502 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 15:06:53.445586 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220}
I20260806 15:06:53.445822 28111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:53.436639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:44759 (local address 127.0.236.65:38923)
0806 15:06:53.436803 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:53.436848 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:53.437942 (+ 1094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:53.438172 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:53.438175 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:53.438485 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:06:53.438737 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:53.438743 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.439653 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.439654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:53.440654 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:53.440658 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.441010 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.441011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:53.441148 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:53.441572 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:06:53.441583 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:06:53.443042 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:06:53.444382 (+ 1340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:53.444395 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:53.444398 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:53.444666 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:53.444880 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:53.444880 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:53.444881 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:53.444986 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:53.445212 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:06:53.445219 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:06:53.445356 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 15:06:53.445475 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 15:06:53.445624 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
I20260806 15:06:53.446261 28108 heartbeater.cc:347] Connected to a master server at 127.0.236.126:44759
I20260806 15:06:53.446353 28108 heartbeater.cc:464] Registering TS with master...
I20260806 15:06:53.446523 28108 heartbeater.cc:511] Master 127.0.236.126:44759 requested a full tablet report, sending...
I20260806 15:06:53.446993 27821 ts_manager.cc:194] Registered new tserver with Master: 3f7bb8d23aea4a2e88ada823fef5b286 (127.0.236.65:42421)
I20260806 15:06:53.447695 27821 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.65@KRBTEST.COM'} at 127.0.236.65:38923
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
I20260806 15:06:53.460845 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:0
--local_ip_for_outbound_sockets=127.0.236.66
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44759
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:32789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:06:53.539417 28118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:53.539604 28118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:53.539624 28118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:53.540743 28118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:53.540783 28118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:53.540812 28118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:06:53.542336 28118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:32789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:53.542553 28118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:53.542743 28118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:53.547308 28124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:53.547327 28127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:53.547328 28125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:53.547596 28118 server_base.cc:1034] running on GCE node
I20260806 15:06:53.547813 28118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:53.548025 28118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:53.549158 28118 hybrid_clock.cc:648] HybridClock initialized: now 1786028813549145 us; error 26 us; skew 500 ppm
Aug 06 15:06:53 dist-test-slave-tc2s krb5kdc[27756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028813, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:06:53.551478 28118 init.cc:377] Logged in from keytab as kudu/127.0.236.66@KRBTEST.COM (short username kudu)
I20260806 15:06:53.551889 28118 webserver.cc:460] Webserver started at http://127.0.236.66:34417/ using document root <none> and password file <none>
I20260806 15:06:53.552098 28118 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:53.552145 28118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:53.552264 28118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:06:53.553198 28118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-1/data/instance:
uuid: "599af0b7d6464c41823ad0a159158281"
format_stamp: "Formatted at 2026-08-06 15:06:53 on dist-test-slave-tc2s"
server_key: "f2819c62e415c31a01664134d7f7148e"
server_key_iv: "bf47470e5a255166344d24f72ca26988"
server_key_version: "encryptionkey@0"
I20260806 15:06:53.553534 28118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-1/wal/instance:
uuid: "599af0b7d6464c41823ad0a159158281"
format_stamp: "Formatted at 2026-08-06 15:06:53 on dist-test-slave-tc2s"
server_key: "f2819c62e415c31a01664134d7f7148e"
server_key_iv: "bf47470e5a255166344d24f72ca26988"
server_key_version: "encryptionkey@0"
I20260806 15:06:53.554828 28118 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:06:53.555480 28139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:53.555619 28118 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:06:53.555698 28118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "599af0b7d6464c41823ad0a159158281"
format_stamp: "Formatted at 2026-08-06 15:06:53 on dist-test-slave-tc2s"
server_key: "f2819c62e415c31a01664134d7f7148e"
server_key_iv: "bf47470e5a255166344d24f72ca26988"
server_key_version: "encryptionkey@0"
I20260806 15:06:53.555765 28118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:53.567005 28118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:53.568907 28118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:53.569042 28118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:53.569222 28118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:06:53.569480 28118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:06:53.569512 28118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:53.569538 28118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:06:53.569555 28118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:53.574988 28118 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:33025
I20260806 15:06:53.575049 28316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:33025 every 8 connection(s)
I20260806 15:06:53.575299 28118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
Aug 06 15:06:53 dist-test-slave-tc2s krb5kdc[27756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028813, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:06:53.583292 27874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:53.575880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:46627 (local address 127.0.236.126:44759)
0806 15:06:53.575942 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:53.575944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:53.576099 (+ 155us) server_negotiation.cc:407] Connection header received
0806 15:06:53.576949 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:53.576951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:53.576963 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:53.577012 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:53.577573 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.578123 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.579070 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.579257 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.579311 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:53.581224 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:53.581226 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:53.581226 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:06:53.581233 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:53.582287 (+ 1054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:53.582633 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:53.582633 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:53.582634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:53.582676 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:53.582844 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:53.582844 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:53.582845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:53.582944 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:53.582987 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:53.583175 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 15:06:53.583229 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 15:06:53.583452 28321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:53.575841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:44759 (local address 127.0.236.66:46627)
0806 15:06:53.576000 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:53.576033 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:53.576845 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:53.577076 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:53.577080 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:53.577311 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:06:53.577487 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:53.577491 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.578208 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.578209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:53.578984 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:53.578987 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.579324 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.579325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:53.579447 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:53.579801 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:06:53.579807 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:06:53.581142 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:06:53.582353 (+ 1211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:53.582359 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:53.582361 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:53.582577 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:53.582718 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:53.582719 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:53.582719 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:53.582804 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:53.582985 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:06:53.582987 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:06:53.583088 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 15:06:53.583159 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 15:06:53.583269 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
I20260806 15:06:53.583859 28317 heartbeater.cc:347] Connected to a master server at 127.0.236.126:44759
I20260806 15:06:53.583942 28317 heartbeater.cc:464] Registering TS with master...
I20260806 15:06:53.584100 28317 heartbeater.cc:511] Master 127.0.236.126:44759 requested a full tablet report, sending...
I20260806 15:06:53.584427 27821 ts_manager.cc:194] Registered new tserver with Master: 599af0b7d6464c41823ad0a159158281 (127.0.236.66:33025)
I20260806 15:06:53.584780 27821 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.66@KRBTEST.COM'} at 127.0.236.66:46627
I20260806 15:06:53.585035 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 28118
I20260806 15:06:53.585136 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-1/wal/instance
I20260806 15:06:53.585304 945 external_mini_cluster.cc:1428] Setting key d8abb648ce3fe9302b4c6b1efddd3ea4
WARNING: no policy specified for kudu/127.0.236.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
I20260806 15:06:53.607681 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:0
--local_ip_for_outbound_sockets=127.0.236.67
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44759
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:32789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:06:53.686173 28325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:53.686367 28325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:53.686398 28325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:53.687505 28325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:53.687543 28325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:53.687567 28325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:06:53.688951 28325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:32789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:53.689110 28325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:53.689278 28325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:53.693768 28335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:53.693781 28334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:53.693774 28338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:53.694059 28325 server_base.cc:1034] running on GCE node
I20260806 15:06:53.694255 28325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:53.694499 28325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:53.695638 28325 hybrid_clock.cc:648] HybridClock initialized: now 1786028813695623 us; error 27 us; skew 500 ppm
Aug 06 15:06:53 dist-test-slave-tc2s krb5kdc[27756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028813, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:06:53.697871 28325 init.cc:377] Logged in from keytab as kudu/127.0.236.67@KRBTEST.COM (short username kudu)
I20260806 15:06:53.698287 28325 webserver.cc:460] Webserver started at http://127.0.236.67:44391/ using document root <none> and password file <none>
I20260806 15:06:53.698457 28325 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:53.698494 28325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:53.698581 28325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:06:53.699496 28325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-2/data/instance:
uuid: "00ff65b5228a48e496a66c0e2ddd4083"
format_stamp: "Formatted at 2026-08-06 15:06:53 on dist-test-slave-tc2s"
server_key: "f701e4e566f93867af5f012f77916f67"
server_key_iv: "712104033d04d0614e8b3742f0923556"
server_key_version: "encryptionkey@0"
I20260806 15:06:53.699764 28325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-2/wal/instance:
uuid: "00ff65b5228a48e496a66c0e2ddd4083"
format_stamp: "Formatted at 2026-08-06 15:06:53 on dist-test-slave-tc2s"
server_key: "f701e4e566f93867af5f012f77916f67"
server_key_iv: "712104033d04d0614e8b3742f0923556"
server_key_version: "encryptionkey@0"
I20260806 15:06:53.700886 28325 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 15:06:53.701532 28348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:53.701689 28325 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:53.701771 28325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "00ff65b5228a48e496a66c0e2ddd4083"
format_stamp: "Formatted at 2026-08-06 15:06:53 on dist-test-slave-tc2s"
server_key: "f701e4e566f93867af5f012f77916f67"
server_key_iv: "712104033d04d0614e8b3742f0923556"
server_key_version: "encryptionkey@0"
I20260806 15:06:53.701822 28325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:53.718567 28325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:53.720506 28325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:53.720603 28325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:53.720757 28325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:06:53.720984 28325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:06:53.721014 28325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:53.721033 28325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:06:53.721050 28325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:53.726442 28325 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:38445
I20260806 15:06:53.726497 28541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:38445 every 8 connection(s)
I20260806 15:06:53.726750 28325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
I20260806 15:06:53.731742 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 28325
I20260806 15:06:53.731832 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-2/wal/instance
I20260806 15:06:53.732004 945 external_mini_cluster.cc:1428] Setting key dd2bcecf4cd3124d85752b055dbb454d
Aug 06 15:06:53 dist-test-slave-tc2s krb5kdc[27756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028813, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:06:53.734956 27874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:53.727385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:57357 (local address 127.0.236.126:44759)
0806 15:06:53.727448 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:53.727450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:53.727588 (+ 138us) server_negotiation.cc:407] Connection header received
0806 15:06:53.728406 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:53.728407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:53.728423 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:53.728470 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:53.728983 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.729478 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.730431 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.730610 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.730653 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:53.732501 (+ 1848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:53.732503 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:53.732503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:06:53.732510 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:53.733708 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:53.734105 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:53.734106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:53.734106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:53.734159 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:53.734372 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:53.734373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:53.734374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:53.734482 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:53.734536 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:53.734783 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 15:06:53.734857 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 15:06:53.735085 28550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:53.727342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:44759 (local address 127.0.236.67:57357)
0806 15:06:53.727486 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:53.727517 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:53.728327 (+ 810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:53.728521 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:53.728524 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:53.728745 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:06:53.728914 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:53.728917 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.729539 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.729540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:53.730369 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:53.730371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.730661 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.730661 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:53.730750 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:53.731115 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:06:53.731123 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:06:53.732419 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:06:53.733780 (+ 1361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:53.733786 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:53.733788 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:53.734038 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:53.734212 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:53.734213 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:53.734213 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:53.734316 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:53.734535 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:06:53.734542 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:06:53.734666 (+ 124us) client_negotiation.cc:769] Sending connection context
0806 15:06:53.734751 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 15:06:53.734889 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
I20260806 15:06:53.735459 28543 heartbeater.cc:347] Connected to a master server at 127.0.236.126:44759
I20260806 15:06:53.735545 28543 heartbeater.cc:464] Registering TS with master...
I20260806 15:06:53.735699 28543 heartbeater.cc:511] Master 127.0.236.126:44759 requested a full tablet report, sending...
I20260806 15:06:53.736017 27821 ts_manager.cc:194] Registered new tserver with Master: 00ff65b5228a48e496a66c0e2ddd4083 (127.0.236.67:38445)
I20260806 15:06:53.736352 27821 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.67@KRBTEST.COM'} at 127.0.236.67:57357
I20260806 15:06:53.743561 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:06:53.750329 27874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:53.744791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51320 (local address 127.0.236.126:44759)
0806 15:06:53.744845 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:53.744847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:53.744950 (+ 103us) server_negotiation.cc:407] Connection header received
0806 15:06:53.745018 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:53.745018 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:53.745027 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:53.745067 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:53.745565 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.746078 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.746759 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.746957 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.747009 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:53.748187 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:53.748188 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:53.748189 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:06:53.748195 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:53.749197 (+ 1002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:53.749547 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:53.749548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:53.749549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:53.749602 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:53.749782 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:53.749783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:53.749783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:53.749957 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:53.750055 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:53.750180 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 15:06:53.750242 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 15:06:53.751654 27821 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51320:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:06:53.753151 27821 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:06:53.759202 28561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:53.755251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49650 (local address 127.0.236.66:33025)
0806 15:06:53.755463 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:53.755467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:53.755477 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:06:53.755512 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:53.755513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:53.755525 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:53.755601 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:53.755805 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.756873 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.758255 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.758779 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.758844 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:53.758902 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:53.759057 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 15:06:53.759105 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":86,"threads_started":1}
I20260806 15:06:53.759228 28560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:53.755154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:38445 (local address 127.0.0.1:57806)
0806 15:06:53.755585 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:53.755594 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:53.755641 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:53.755895 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:53.755896 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:53.755898 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:53.755935 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:53.755936 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.756812 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.756813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:53.758141 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:53.758145 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.759009 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.759011 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:53.759064 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:53.759067 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:06:53.759111 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 15:06:53.759165 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":2,"thread_start_us":43,"threads_started":1}
I20260806 15:06:53.759277 28562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:53.755342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57806 (local address 127.0.236.67:38445)
0806 15:06:53.755738 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:53.755742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:53.755751 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:06:53.755783 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:53.755785 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:53.755800 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:53.756004 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:53.756076 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.756720 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.758219 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.758930 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.759000 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:53.759059 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:53.759124 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 15:06:53.759161 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":67,"threads_started":1}
I20260806 15:06:53.759581 28559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:53.755145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:33025 (local address 127.0.0.1:49650)
0806 15:06:53.755294 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:53.755329 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:53.755391 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:53.755623 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:53.755627 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:53.755629 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:53.755700 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:53.755702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.756974 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.756975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:53.758167 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:53.758169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.758855 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.758856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:53.758936 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:53.758949 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 15:06:53.759490 (+ 541us) client_negotiation.cc:240] Negotiation successful
0806 15:06:53.759533 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":46,"thread_start_us":51,"threads_started":1}
I20260806 15:06:53.759898 28429 tablet_service.cc:1467] Processing CreateTablet for tablet bca58768da324eeb92e852d63c477db2 (DEFAULT_TABLE table=test-table [id=741fea0e2fd046fbafb9bafd3f871055]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:06:53.760249 28429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bca58768da324eeb92e852d63c477db2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:53.760293 28218 tablet_service.cc:1467] Processing CreateTablet for tablet bca58768da324eeb92e852d63c477db2 (DEFAULT_TABLE table=test-table [id=741fea0e2fd046fbafb9bafd3f871055]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:06:53.760627 28218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bca58768da324eeb92e852d63c477db2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:53.763448 28565 tablet_bootstrap.cc:492] T bca58768da324eeb92e852d63c477db2 P 00ff65b5228a48e496a66c0e2ddd4083: Bootstrap starting.
I20260806 15:06:53.763676 28557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:53.755160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:42421 (local address 127.0.0.1:45324)
0806 15:06:53.755690 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:53.755696 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:53.755730 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:53.756041 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:53.756042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:53.756044 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:53.756084 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:53.756085 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.757204 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.757206 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:53.758173 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:53.758176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.759115 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.759116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:53.759174 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:53.759178 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:06:53.759224 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 15:06:53.759260 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":36,"thread_start_us":64,"threads_started":1}
I20260806 15:06:53.763947 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:53.755390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45324 (local address 127.0.236.65:42421)
0806 15:06:53.755556 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:53.755560 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:53.755745 (+ 185us) server_negotiation.cc:407] Connection header received
0806 15:06:53.755860 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:53.755862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:53.755878 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:53.755957 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:53.756156 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.757119 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.758257 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.758977 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.759032 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:53.759080 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:53.763833 (+ 4753us) server_negotiation.cc:299] Negotiation successful
0806 15:06:53.763881 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
I20260806 15:06:53.764094 28565 tablet_bootstrap.cc:654] T bca58768da324eeb92e852d63c477db2 P 00ff65b5228a48e496a66c0e2ddd4083: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:53.764125 28566 tablet_bootstrap.cc:492] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281: Bootstrap starting.
I20260806 15:06:53.764624 28566 tablet_bootstrap.cc:654] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:53.764627 28565 log.cc:824] T bca58768da324eeb92e852d63c477db2 P 00ff65b5228a48e496a66c0e2ddd4083: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:53.764657 28003 tablet_service.cc:1467] Processing CreateTablet for tablet bca58768da324eeb92e852d63c477db2 (DEFAULT_TABLE table=test-table [id=741fea0e2fd046fbafb9bafd3f871055]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:06:53.764973 28003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bca58768da324eeb92e852d63c477db2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:53.765081 28566 log.cc:824] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:53.765502 28565 tablet_bootstrap.cc:492] T bca58768da324eeb92e852d63c477db2 P 00ff65b5228a48e496a66c0e2ddd4083: No bootstrap required, opened a new log
I20260806 15:06:53.765560 28565 ts_tablet_manager.cc:1397] T bca58768da324eeb92e852d63c477db2 P 00ff65b5228a48e496a66c0e2ddd4083: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:06:53.765686 28566 tablet_bootstrap.cc:492] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281: No bootstrap required, opened a new log
I20260806 15:06:53.765727 28566 ts_tablet_manager.cc:1397] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:06:53.767046 28566 raft_consensus.cc:348] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "599af0b7d6464c41823ad0a159158281" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 33025 } } peers { permanent_uuid: "00ff65b5228a48e496a66c0e2ddd4083" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38445 } } peers { permanent_uuid: "3f7bb8d23aea4a2e88ada823fef5b286" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 42421 } }
I20260806 15:06:53.767180 28566 raft_consensus.cc:374] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:53.767213 28566 raft_consensus.cc:729] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 599af0b7d6464c41823ad0a159158281, State: Initialized, Role: FOLLOWER
I20260806 15:06:53.767359 28566 consensus_queue.cc:260] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "599af0b7d6464c41823ad0a159158281" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 33025 } } peers { permanent_uuid: "00ff65b5228a48e496a66c0e2ddd4083" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38445 } } peers { permanent_uuid: "3f7bb8d23aea4a2e88ada823fef5b286" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 42421 } }
I20260806 15:06:53.767380 28565 raft_consensus.cc:348] T bca58768da324eeb92e852d63c477db2 P 00ff65b5228a48e496a66c0e2ddd4083 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "599af0b7d6464c41823ad0a159158281" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 33025 } } peers { permanent_uuid: "00ff65b5228a48e496a66c0e2ddd4083" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38445 } } peers { permanent_uuid: "3f7bb8d23aea4a2e88ada823fef5b286" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 42421 } }
I20260806 15:06:53.767549 28565 raft_consensus.cc:374] T bca58768da324eeb92e852d63c477db2 P 00ff65b5228a48e496a66c0e2ddd4083 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:53.767592 28565 raft_consensus.cc:729] T bca58768da324eeb92e852d63c477db2 P 00ff65b5228a48e496a66c0e2ddd4083 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00ff65b5228a48e496a66c0e2ddd4083, State: Initialized, Role: FOLLOWER
I20260806 15:06:53.767616 28566 ts_tablet_manager.cc:1428] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 15:06:53.767735 28317 heartbeater.cc:503] Master 127.0.236.126:44759 was elected leader, sending a full tablet report...
I20260806 15:06:53.767701 28565 consensus_queue.cc:260] T bca58768da324eeb92e852d63c477db2 P 00ff65b5228a48e496a66c0e2ddd4083 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "599af0b7d6464c41823ad0a159158281" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 33025 } } peers { permanent_uuid: "00ff65b5228a48e496a66c0e2ddd4083" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38445 } } peers { permanent_uuid: "3f7bb8d23aea4a2e88ada823fef5b286" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 42421 } }
I20260806 15:06:53.767975 28543 heartbeater.cc:503] Master 127.0.236.126:44759 was elected leader, sending a full tablet report...
I20260806 15:06:53.767980 28565 ts_tablet_manager.cc:1428] T bca58768da324eeb92e852d63c477db2 P 00ff65b5228a48e496a66c0e2ddd4083: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 15:06:53.768167 28575 tablet_bootstrap.cc:492] T bca58768da324eeb92e852d63c477db2 P 3f7bb8d23aea4a2e88ada823fef5b286: Bootstrap starting.
I20260806 15:06:53.769057 28575 tablet_bootstrap.cc:654] T bca58768da324eeb92e852d63c477db2 P 3f7bb8d23aea4a2e88ada823fef5b286: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:53.770012 28575 log.cc:824] T bca58768da324eeb92e852d63c477db2 P 3f7bb8d23aea4a2e88ada823fef5b286: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:53.771154 28575 tablet_bootstrap.cc:492] T bca58768da324eeb92e852d63c477db2 P 3f7bb8d23aea4a2e88ada823fef5b286: No bootstrap required, opened a new log
I20260806 15:06:53.771232 28575 ts_tablet_manager.cc:1397] T bca58768da324eeb92e852d63c477db2 P 3f7bb8d23aea4a2e88ada823fef5b286: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 15:06:53.773855 28575 raft_consensus.cc:348] T bca58768da324eeb92e852d63c477db2 P 3f7bb8d23aea4a2e88ada823fef5b286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "599af0b7d6464c41823ad0a159158281" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 33025 } } peers { permanent_uuid: "00ff65b5228a48e496a66c0e2ddd4083" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38445 } } peers { permanent_uuid: "3f7bb8d23aea4a2e88ada823fef5b286" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 42421 } }
I20260806 15:06:53.774250 28575 raft_consensus.cc:374] T bca58768da324eeb92e852d63c477db2 P 3f7bb8d23aea4a2e88ada823fef5b286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:53.774323 28575 raft_consensus.cc:729] T bca58768da324eeb92e852d63c477db2 P 3f7bb8d23aea4a2e88ada823fef5b286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f7bb8d23aea4a2e88ada823fef5b286, State: Initialized, Role: FOLLOWER
I20260806 15:06:53.774513 28575 consensus_queue.cc:260] T bca58768da324eeb92e852d63c477db2 P 3f7bb8d23aea4a2e88ada823fef5b286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "599af0b7d6464c41823ad0a159158281" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 33025 } } peers { permanent_uuid: "00ff65b5228a48e496a66c0e2ddd4083" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38445 } } peers { permanent_uuid: "3f7bb8d23aea4a2e88ada823fef5b286" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 42421 } }
I20260806 15:06:53.774976 28575 ts_tablet_manager.cc:1428] T bca58768da324eeb92e852d63c477db2 P 3f7bb8d23aea4a2e88ada823fef5b286: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260806 15:06:53.774993 28108 heartbeater.cc:503] Master 127.0.236.126:44759 was elected leader, sending a full tablet report...
W20260806 15:06:53.825759 28319 tablet.cc:2369] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 15:06:53.936631 28109 tablet.cc:2369] T bca58768da324eeb92e852d63c477db2 P 3f7bb8d23aea4a2e88ada823fef5b286: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:06:53.952896 28572 raft_consensus.cc:482] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:06:53.953011 28572 raft_consensus.cc:504] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "599af0b7d6464c41823ad0a159158281" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 33025 } } peers { permanent_uuid: "00ff65b5228a48e496a66c0e2ddd4083" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38445 } } peers { permanent_uuid: "3f7bb8d23aea4a2e88ada823fef5b286" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 42421 } }
I20260806 15:06:53.953372 28572 leader_election.cc:288] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 00ff65b5228a48e496a66c0e2ddd4083 (127.0.236.67:38445), 3f7bb8d23aea4a2e88ada823fef5b286 (127.0.236.65:42421)
I20260806 15:06:53.956074 28321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:53.953522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:38445 (local address 127.0.236.66:50655)
0806 15:06:53.953561 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:53.953571 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:53.953608 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:53.953798 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:53.953800 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:53.953802 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:53.953872 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:53.953874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.954550 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.954551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:53.955299 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:53.955302 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.955846 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.955847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:53.955917 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:53.955920 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:06:53.955965 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 15:06:53.956003 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19,"mutex_wait_us":1}
I20260806 15:06:53.956073 28562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:53.953613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:50655 (local address 127.0.236.67:38445)
0806 15:06:53.953670 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:53.953672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:53.953680 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:06:53.953704 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:53.953705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:53.953715 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:53.953775 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:53.953957 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.954445 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.955395 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.955763 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.955827 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:53.955836 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:53.955976 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 15:06:53.956007 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 15:06:53.956178 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:53.953587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:55139 (local address 127.0.236.65:42421)
0806 15:06:53.953636 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:53.953638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:53.953875 (+ 237us) server_negotiation.cc:407] Connection header received
0806 15:06:53.953936 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:53.953937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:53.953948 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:53.953994 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:53.954116 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.954655 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.955534 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.955880 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.955938 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:53.955946 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:53.956081 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 15:06:53.956116 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 15:06:53.956187 28580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:53.953521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:42421 (local address 127.0.236.66:55139)
0806 15:06:53.953813 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:53.953820 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:53.953867 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:53.953995 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:53.953996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:53.953998 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:53.954063 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:53.954064 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.954718 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.954718 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:53.955454 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:53.955457 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.955950 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.955951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:53.956023 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:53.956027 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:06:53.956070 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 15:06:53.956107 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":132,"thread_start_us":104,"threads_started":1}
I20260806 15:06:53.956449 28462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bca58768da324eeb92e852d63c477db2" candidate_uuid: "599af0b7d6464c41823ad0a159158281" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00ff65b5228a48e496a66c0e2ddd4083" is_pre_election: true
I20260806 15:06:53.956595 28462 raft_consensus.cc:2454] T bca58768da324eeb92e852d63c477db2 P 00ff65b5228a48e496a66c0e2ddd4083 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 599af0b7d6464c41823ad0a159158281 in term 0.
I20260806 15:06:53.956563 28038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bca58768da324eeb92e852d63c477db2" candidate_uuid: "599af0b7d6464c41823ad0a159158281" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f7bb8d23aea4a2e88ada823fef5b286" is_pre_election: true
I20260806 15:06:53.956693 28038 raft_consensus.cc:2454] T bca58768da324eeb92e852d63c477db2 P 3f7bb8d23aea4a2e88ada823fef5b286 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 599af0b7d6464c41823ad0a159158281 in term 0.
I20260806 15:06:53.956815 28146 leader_election.cc:302] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00ff65b5228a48e496a66c0e2ddd4083, 599af0b7d6464c41823ad0a159158281; no voters:
I20260806 15:06:53.956933 28572 raft_consensus.cc:2781] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:06:53.956995 28572 raft_consensus.cc:482] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:06:53.957010 28572 raft_consensus.cc:3037] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:53.957652 28572 raft_consensus.cc:504] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "599af0b7d6464c41823ad0a159158281" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 33025 } } peers { permanent_uuid: "00ff65b5228a48e496a66c0e2ddd4083" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38445 } } peers { permanent_uuid: "3f7bb8d23aea4a2e88ada823fef5b286" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 42421 } }
I20260806 15:06:53.957755 28572 leader_election.cc:288] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281 [CANDIDATE]: Term 1 election: Requested vote from peers 00ff65b5228a48e496a66c0e2ddd4083 (127.0.236.67:38445), 3f7bb8d23aea4a2e88ada823fef5b286 (127.0.236.65:42421)
I20260806 15:06:53.957975 28462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bca58768da324eeb92e852d63c477db2" candidate_uuid: "599af0b7d6464c41823ad0a159158281" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00ff65b5228a48e496a66c0e2ddd4083"
I20260806 15:06:53.957979 28038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bca58768da324eeb92e852d63c477db2" candidate_uuid: "599af0b7d6464c41823ad0a159158281" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f7bb8d23aea4a2e88ada823fef5b286"
I20260806 15:06:53.958040 28462 raft_consensus.cc:3037] T bca58768da324eeb92e852d63c477db2 P 00ff65b5228a48e496a66c0e2ddd4083 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:53.958065 28038 raft_consensus.cc:3037] T bca58768da324eeb92e852d63c477db2 P 3f7bb8d23aea4a2e88ada823fef5b286 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:53.958693 28462 raft_consensus.cc:2454] T bca58768da324eeb92e852d63c477db2 P 00ff65b5228a48e496a66c0e2ddd4083 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 599af0b7d6464c41823ad0a159158281 in term 1.
I20260806 15:06:53.958721 28038 raft_consensus.cc:2454] T bca58768da324eeb92e852d63c477db2 P 3f7bb8d23aea4a2e88ada823fef5b286 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 599af0b7d6464c41823ad0a159158281 in term 1.
I20260806 15:06:53.958874 28147 leader_election.cc:302] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f7bb8d23aea4a2e88ada823fef5b286, 599af0b7d6464c41823ad0a159158281; no voters:
I20260806 15:06:53.958972 28572 raft_consensus.cc:2781] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:06:53.959091 28572 raft_consensus.cc:686] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281 [term 1 LEADER]: Becoming Leader. State: Replica: 599af0b7d6464c41823ad0a159158281, State: Running, Role: LEADER
I20260806 15:06:53.959172 28572 consensus_queue.cc:237] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "599af0b7d6464c41823ad0a159158281" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 33025 } } peers { permanent_uuid: "00ff65b5228a48e496a66c0e2ddd4083" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38445 } } peers { permanent_uuid: "3f7bb8d23aea4a2e88ada823fef5b286" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 42421 } }
I20260806 15:06:53.959950 27821 catalog_manager.cc:5495] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281 reported cstate change: term changed from 0 to 1, leader changed from <none> to 599af0b7d6464c41823ad0a159158281 (127.0.236.66). New cstate: current_term: 1 leader_uuid: "599af0b7d6464c41823ad0a159158281" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "599af0b7d6464c41823ad0a159158281" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 33025 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00ff65b5228a48e496a66c0e2ddd4083" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f7bb8d23aea4a2e88ada823fef5b286" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 42421 } health_report { overall_health: UNKNOWN } } }
W20260806 15:06:53.977219 28545 tablet.cc:2369] T bca58768da324eeb92e852d63c477db2 P 00ff65b5228a48e496a66c0e2ddd4083: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:06:53.984422 28561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:53.982548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49654 (local address 127.0.236.66:33025)
0806 15:06:53.982595 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:53.982597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:53.982605 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:06:53.982647 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:53.982647 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:53.982661 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:53.982709 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:06:53.982806 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.983234 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.983878 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:53.984044 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:53.984084 (+ 40us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:53.984173 (+ 89us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:06:53.984251 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:06:53.984312 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:53.984345 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 15:06:53.984360 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 15:06:53.985968 28462 raft_consensus.cc:1261] T bca58768da324eeb92e852d63c477db2 P 00ff65b5228a48e496a66c0e2ddd4083 [term 1 FOLLOWER]: Refusing update from remote peer 599af0b7d6464c41823ad0a159158281: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:06:53.985971 28038 raft_consensus.cc:1261] T bca58768da324eeb92e852d63c477db2 P 3f7bb8d23aea4a2e88ada823fef5b286 [term 1 FOLLOWER]: Refusing update from remote peer 599af0b7d6464c41823ad0a159158281: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:06:53.986250 28583 consensus_queue.cc:1035] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f7bb8d23aea4a2e88ada823fef5b286" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 42421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:06:53.986363 28572 consensus_queue.cc:1035] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00ff65b5228a48e496a66c0e2ddd4083" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:06:53.988381 28588 mvcc.cc:205] Tried to move back new op lower bound from 7315574022084227072 to 7315574021977198592. Current Snapshot: MvccSnapshot[applied={T|T < 7315574022084227072}]
I20260806 15:06:53.988946 28589 mvcc.cc:205] Tried to move back new op lower bound from 7315574022084227072 to 7315574021977198592. Current Snapshot: MvccSnapshot[applied={T|T < 7315574022084227072 or (T in {7315574022084227072})}]
I20260806 15:06:53.990502 27819 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:06:53.990581 27819 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:06:53.991261 27819 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c466d15aa7149cdb4b113926549ac0c: Sending DeleteTablet for 3 replicas of tablet bca58768da324eeb92e852d63c477db2
I20260806 15:06:53.991585 28218 tablet_service.cc:1514] Processing DeleteTablet for tablet bca58768da324eeb92e852d63c477db2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:06:53 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:49650
I20260806 15:06:53.991600 28429 tablet_service.cc:1514] Processing DeleteTablet for tablet bca58768da324eeb92e852d63c477db2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:06:53 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:57806
I20260806 15:06:53.991809 28003 tablet_service.cc:1514] Processing DeleteTablet for tablet bca58768da324eeb92e852d63c477db2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:06:53 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:45324
I20260806 15:06:53.992005 28609 tablet_replica.cc:330] T bca58768da324eeb92e852d63c477db2 P 3f7bb8d23aea4a2e88ada823fef5b286: stopping tablet replica
I20260806 15:06:53.992025 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 27913
I20260806 15:06:53.992096 28609 raft_consensus.cc:2229] T bca58768da324eeb92e852d63c477db2 P 3f7bb8d23aea4a2e88ada823fef5b286 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:06:53.992193 28609 raft_consensus.cc:2258] T bca58768da324eeb92e852d63c477db2 P 3f7bb8d23aea4a2e88ada823fef5b286 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:06:53.992332 28605 tablet_replica.cc:330] T bca58768da324eeb92e852d63c477db2 P 00ff65b5228a48e496a66c0e2ddd4083: stopping tablet replica
I20260806 15:06:53.992400 28605 raft_consensus.cc:2229] T bca58768da324eeb92e852d63c477db2 P 00ff65b5228a48e496a66c0e2ddd4083 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:06:53.992470 28605 raft_consensus.cc:2258] T bca58768da324eeb92e852d63c477db2 P 00ff65b5228a48e496a66c0e2ddd4083 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:06:53.992601 28608 tablet_replica.cc:330] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281: stopping tablet replica
I20260806 15:06:53.992683 28608 raft_consensus.cc:2229] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281 [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:06:53.992756 28608 raft_consensus.cc:2258] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:06:53.995153 28605 ts_tablet_manager.cc:1905] T bca58768da324eeb92e852d63c477db2 P 00ff65b5228a48e496a66c0e2ddd4083: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:06:53.995159 28608 ts_tablet_manager.cc:1905] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:06:53.996887 28608 ts_tablet_manager.cc:1918] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:06:53.996976 28608 log.cc:1196] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-1/wal/wals/bca58768da324eeb92e852d63c477db2
I20260806 15:06:53.996910 28605 ts_tablet_manager.cc:1918] T bca58768da324eeb92e852d63c477db2 P 00ff65b5228a48e496a66c0e2ddd4083: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:06:53.997169 28605 log.cc:1196] T bca58768da324eeb92e852d63c477db2 P 00ff65b5228a48e496a66c0e2ddd4083: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/bca58768da324eeb92e852d63c477db2
I20260806 15:06:53.997247 28608 ts_tablet_manager.cc:1939] T bca58768da324eeb92e852d63c477db2 P 599af0b7d6464c41823ad0a159158281: Deleting consensus metadata
I20260806 15:06:53.997575 28605 ts_tablet_manager.cc:1939] T bca58768da324eeb92e852d63c477db2 P 00ff65b5228a48e496a66c0e2ddd4083: Deleting consensus metadata
I20260806 15:06:53.997627 27796 catalog_manager.cc:4841] TS 599af0b7d6464c41823ad0a159158281 (127.0.236.66:33025): tablet bca58768da324eeb92e852d63c477db2 (table test-table [id=741fea0e2fd046fbafb9bafd3f871055]) successfully deleted
I20260806 15:06:53.998674 27798 catalog_manager.cc:4841] TS 00ff65b5228a48e496a66c0e2ddd4083 (127.0.236.67:38445): tablet bca58768da324eeb92e852d63c477db2 (table test-table [id=741fea0e2fd046fbafb9bafd3f871055]) successfully deleted
W20260806 15:06:54.003506 27799 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.236.65:42421 (error 108)
I20260806 15:06:54.003824 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 28118
I20260806 15:06:54.003996 28557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:54.003859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:42421 (local address 127.0.0.1:45340)
0806 15:06:54.003929 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:54.003965 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.65:42421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 15:06:54.006237 27799 catalog_manager.cc:4584] TS 3f7bb8d23aea4a2e88ada823fef5b286 (127.0.236.65:42421): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bca58768da324eeb92e852d63c477db2: Network error: Client connection negotiation failed: client connection to 127.0.236.65:42421: connect: Connection refused (error 111)
I20260806 15:06:54.011128 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 28325
I20260806 15:06:54.018814 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 27772
2026-08-06T15:06:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2078 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T15:06:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:06:54Z Disabled control of system clock
I20260806 15:06:54.051637 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:45337
--webserver_interface=127.0.236.126
--webserver_port=0
--builtin_ntp_servers=127.0.236.84:42659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:45337
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 15:06:54.129781 28617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:54.129987 28617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:54.130007 28617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:54.131099 28617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:06:54.131139 28617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:54.131150 28617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:54.131171 28617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:06:54.131194 28617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:06:54.132510 28617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:42659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:45337
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:45337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:54.132720 28617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:54.132977 28617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:54.137640 28628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:54.137676 28624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:54.137821 28617 server_base.cc:1034] running on GCE node
W20260806 15:06:54.137657 28623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:54.138175 28617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:54.138427 28617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:54.139564 28617 hybrid_clock.cc:648] HybridClock initialized: now 1786028814139551 us; error 29 us; skew 500 ppm
I20260806 15:06:54.140656 28617 webserver.cc:460] Webserver started at http://127.0.236.126:42471/ using document root <none> and password file <none>
I20260806 15:06:54.140879 28617 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:54.140923 28617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:54.141057 28617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:06:54.142062 28617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "db13968d82754af89ccb6d5333137cda"
format_stamp: "Formatted at 2026-08-06 15:06:54 on dist-test-slave-tc2s"
server_key: "31f177a8db1bbbdd79af843543a02ac9"
server_key_iv: "d80b95fc8f088593347fc4aefaebc796"
server_key_version: "encryptionkey@0"
I20260806 15:06:54.142369 28617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "db13968d82754af89ccb6d5333137cda"
format_stamp: "Formatted at 2026-08-06 15:06:54 on dist-test-slave-tc2s"
server_key: "31f177a8db1bbbdd79af843543a02ac9"
server_key_iv: "d80b95fc8f088593347fc4aefaebc796"
server_key_version: "encryptionkey@0"
I20260806 15:06:54.143491 28617 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:06:54.144135 28639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:54.144259 28617 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:54.144336 28617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "db13968d82754af89ccb6d5333137cda"
format_stamp: "Formatted at 2026-08-06 15:06:54 on dist-test-slave-tc2s"
server_key: "31f177a8db1bbbdd79af843543a02ac9"
server_key_iv: "d80b95fc8f088593347fc4aefaebc796"
server_key_version: "encryptionkey@0"
I20260806 15:06:54.144402 28617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:54.159301 28617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:54.159538 28617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:54.159657 28617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:54.163257 28617 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:45337
I20260806 15:06:54.163288 28729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:45337 every 8 connection(s)
I20260806 15:06:54.163566 28617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:06:54.164422 28730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:54.165458 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 28617
I20260806 15:06:54.165534 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:06:54.165707 945 external_mini_cluster.cc:1428] Setting key 1bdb5d82f13191f75385ae1f698a00e3
I20260806 15:06:54.166815 28730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db13968d82754af89ccb6d5333137cda: Bootstrap starting.
I20260806 15:06:54.167484 28730 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db13968d82754af89ccb6d5333137cda: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:54.167784 28730 log.cc:824] T 00000000000000000000000000000000 P db13968d82754af89ccb6d5333137cda: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:54.168465 28730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db13968d82754af89ccb6d5333137cda: No bootstrap required, opened a new log
I20260806 15:06:54.170521 28730 raft_consensus.cc:348] T 00000000000000000000000000000000 P db13968d82754af89ccb6d5333137cda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db13968d82754af89ccb6d5333137cda" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 45337 } }
I20260806 15:06:54.170680 28730 raft_consensus.cc:374] T 00000000000000000000000000000000 P db13968d82754af89ccb6d5333137cda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:54.170727 28730 raft_consensus.cc:729] T 00000000000000000000000000000000 P db13968d82754af89ccb6d5333137cda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db13968d82754af89ccb6d5333137cda, State: Initialized, Role: FOLLOWER
I20260806 15:06:54.170849 28730 consensus_queue.cc:260] T 00000000000000000000000000000000 P db13968d82754af89ccb6d5333137cda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db13968d82754af89ccb6d5333137cda" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 45337 } }
I20260806 15:06:54.170939 28730 raft_consensus.cc:388] T 00000000000000000000000000000000 P db13968d82754af89ccb6d5333137cda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:06:54.170971 28730 raft_consensus.cc:482] T 00000000000000000000000000000000 P db13968d82754af89ccb6d5333137cda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:06:54.171007 28730 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db13968d82754af89ccb6d5333137cda [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:54.171168 28737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:54.166155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55636 (local address 127.0.236.126:45337)
0806 15:06:54.166352 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:54.166361 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:54.166393 (+ 32us) server_negotiation.cc:407] Connection header received
0806 15:06:54.166867 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:54.166876 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:54.166949 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:54.167089 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:54.167244 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.168446 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.169208 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.169812 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.169966 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:54.170509 (+ 543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:54.170513 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:54.170514 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:06:54.170697 (+ 183us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:54.170738 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:06:54.170752 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:54.170815 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:54.170904 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 15:06:54.170980 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
I20260806 15:06:54.171582 28730 raft_consensus.cc:504] T 00000000000000000000000000000000 P db13968d82754af89ccb6d5333137cda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db13968d82754af89ccb6d5333137cda" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 45337 } }
I20260806 15:06:54.171671 28730 leader_election.cc:302] T 00000000000000000000000000000000 P db13968d82754af89ccb6d5333137cda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db13968d82754af89ccb6d5333137cda; no voters:
I20260806 15:06:54.171829 28730 leader_election.cc:288] T 00000000000000000000000000000000 P db13968d82754af89ccb6d5333137cda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:06:54.171854 28739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db13968d82754af89ccb6d5333137cda [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:06:54.171964 28739 raft_consensus.cc:686] T 00000000000000000000000000000000 P db13968d82754af89ccb6d5333137cda [term 1 LEADER]: Becoming Leader. State: Replica: db13968d82754af89ccb6d5333137cda, State: Running, Role: LEADER
I20260806 15:06:54.172087 28730 sys_catalog.cc:564] T 00000000000000000000000000000000 P db13968d82754af89ccb6d5333137cda [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:06:54.172079 28739 consensus_queue.cc:237] T 00000000000000000000000000000000 P db13968d82754af89ccb6d5333137cda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db13968d82754af89ccb6d5333137cda" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 45337 } }
I20260806 15:06:54.172456 28741 sys_catalog.cc:455] T 00000000000000000000000000000000 P db13968d82754af89ccb6d5333137cda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db13968d82754af89ccb6d5333137cda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db13968d82754af89ccb6d5333137cda" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 45337 } } }
I20260806 15:06:54.172467 28744 sys_catalog.cc:455] T 00000000000000000000000000000000 P db13968d82754af89ccb6d5333137cda [sys.catalog]: SysCatalogTable state changed. Reason: New leader db13968d82754af89ccb6d5333137cda. Latest consensus state: current_term: 1 leader_uuid: "db13968d82754af89ccb6d5333137cda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db13968d82754af89ccb6d5333137cda" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 45337 } } }
I20260806 15:06:54.172622 28741 sys_catalog.cc:458] T 00000000000000000000000000000000 P db13968d82754af89ccb6d5333137cda [sys.catalog]: This master's current role is: LEADER
I20260806 15:06:54.172714 28744 sys_catalog.cc:458] T 00000000000000000000000000000000 P db13968d82754af89ccb6d5333137cda [sys.catalog]: This master's current role is: LEADER
I20260806 15:06:54.173019 28753 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:06:54.173481 28753 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:06:54.174856 28753 catalog_manager.cc:1347] Generated new cluster ID: eececf69b48b421ca82a0da2f4d66ef5
I20260806 15:06:54.174903 28753 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:06:54.185261 28753 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:06:54.185714 28753 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:06:54.200906 28753 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db13968d82754af89ccb6d5333137cda: Generated new TSK 0
I20260806 15:06:54.201056 28753 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:06:54.205293 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:0
--local_ip_for_outbound_sockets=127.0.236.65
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:45337
--builtin_ntp_servers=127.0.236.84:42659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 15:06:54.320995 28766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:54.321271 28766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:54.321308 28766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:54.323019 28766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:54.323077 28766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:54.323112 28766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:06:54.325079 28766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:42659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:45337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:54.325304 28766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:54.325606 28766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:54.330513 28773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:54.330518 28777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:54.330555 28780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:54.331629 28766 server_base.cc:1034] running on GCE node
I20260806 15:06:54.331983 28766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:54.332262 28766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:54.333415 28766 hybrid_clock.cc:648] HybridClock initialized: now 1786028814333407 us; error 39 us; skew 500 ppm
I20260806 15:06:54.334599 28766 webserver.cc:460] Webserver started at http://127.0.236.65:39151/ using document root <none> and password file <none>
I20260806 15:06:54.334808 28766 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:54.334869 28766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:54.334998 28766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:06:54.335983 28766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-0/data/instance:
uuid: "03def777f96149f8b5575e9e61b4262f"
format_stamp: "Formatted at 2026-08-06 15:06:54 on dist-test-slave-tc2s"
server_key: "4b27476959074627b0b7443d98c61e30"
server_key_iv: "01f7c708efda94c9e799425418a53b5f"
server_key_version: "encryptionkey@0"
I20260806 15:06:54.336298 28766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-0/wal/instance:
uuid: "03def777f96149f8b5575e9e61b4262f"
format_stamp: "Formatted at 2026-08-06 15:06:54 on dist-test-slave-tc2s"
server_key: "4b27476959074627b0b7443d98c61e30"
server_key_iv: "01f7c708efda94c9e799425418a53b5f"
server_key_version: "encryptionkey@0"
I20260806 15:06:54.337455 28766 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:06:54.338124 28787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:54.338253 28766 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:54.338335 28766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "03def777f96149f8b5575e9e61b4262f"
format_stamp: "Formatted at 2026-08-06 15:06:54 on dist-test-slave-tc2s"
server_key: "4b27476959074627b0b7443d98c61e30"
server_key_iv: "01f7c708efda94c9e799425418a53b5f"
server_key_version: "encryptionkey@0"
I20260806 15:06:54.338393 28766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:54.353976 28766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:54.354214 28766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:54.354331 28766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:54.354521 28766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:06:54.354807 28766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:06:54.354854 28766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:54.354882 28766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:06:54.354905 28766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:54.360069 28766 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:45555
I20260806 15:06:54.360131 28956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:45555 every 8 connection(s)
I20260806 15:06:54.360396 28766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:06:54.361199 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 28766
I20260806 15:06:54.361291 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-0/wal/instance
I20260806 15:06:54.361470 945 external_mini_cluster.cc:1428] Setting key 610d6d43732d6c0d9a9d6e17b2ec341a
I20260806 15:06:54.362418 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:0
--local_ip_for_outbound_sockets=127.0.236.66
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:45337
--builtin_ntp_servers=127.0.236.84:42659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 15:06:54.365491 28737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:54.360976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:36249 (local address 127.0.236.126:45337)
0806 15:06:54.361031 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:54.361033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:54.361279 (+ 246us) server_negotiation.cc:407] Connection header received
0806 15:06:54.361836 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:54.361838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:54.361855 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:54.361924 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:54.362646 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.363291 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.364204 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.364486 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.364557 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:54.365242 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:54.365244 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:54.365244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:06:54.365249 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:54.365265 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:06:54.365269 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:54.365316 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:54.365388 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 15:06:54.365414 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 15:06:54.365700 28960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:54.360975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:45337 (local address 127.0.236.65:36249)
0806 15:06:54.361159 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:54.361196 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:54.361736 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:54.361988 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:54.361991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:54.362000 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:06:54.362563 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:54.362570 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.363366 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.363367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:54.364126 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:54.364128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.364556 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.364557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:54.364653 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:54.365118 (+ 465us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:06:54.365118 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:06:54.365144 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:06:54.365163 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:06:54.365322 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:06:54.365329 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:06:54.365331 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 15:06:54.365392 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 15:06:54.365499 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
I20260806 15:06:54.366134 28958 heartbeater.cc:347] Connected to a master server at 127.0.236.126:45337
I20260806 15:06:54.366216 28958 heartbeater.cc:464] Registering TS with master...
I20260806 15:06:54.366369 28958 heartbeater.cc:511] Master 127.0.236.126:45337 requested a full tablet report, sending...
I20260806 15:06:54.366817 28670 ts_manager.cc:194] Registered new tserver with Master: 03def777f96149f8b5575e9e61b4262f (127.0.236.65:45555)
I20260806 15:06:54.367424 28670 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.236.65:36249
W20260806 15:06:54.443840 28961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:54.444020 28961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:54.444049 28961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:54.445129 28961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:54.445168 28961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:54.445199 28961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:06:54.446552 28961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:42659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:45337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:54.446770 28961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:54.447028 28961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 15:06:54.451637 28961 server_base.cc:1034] running on GCE node
W20260806 15:06:54.451578 28973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:54.451623 28977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:54.451577 28975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:54.451992 28961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:54.452209 28961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:54.453332 28961 hybrid_clock.cc:648] HybridClock initialized: now 1786028814453320 us; error 31 us; skew 500 ppm
I20260806 15:06:54.454463 28961 webserver.cc:460] Webserver started at http://127.0.236.66:40227/ using document root <none> and password file <none>
I20260806 15:06:54.454681 28961 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:54.454725 28961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:54.454846 28961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:06:54.455780 28961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-1/data/instance:
uuid: "de66286beadc49cd9ac6396c82cc374e"
format_stamp: "Formatted at 2026-08-06 15:06:54 on dist-test-slave-tc2s"
server_key: "c6ddc27b255aed7d40c1300f48b5cac2"
server_key_iv: "b0659bc821038efa117205df8372d05b"
server_key_version: "encryptionkey@0"
I20260806 15:06:54.456084 28961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-1/wal/instance:
uuid: "de66286beadc49cd9ac6396c82cc374e"
format_stamp: "Formatted at 2026-08-06 15:06:54 on dist-test-slave-tc2s"
server_key: "c6ddc27b255aed7d40c1300f48b5cac2"
server_key_iv: "b0659bc821038efa117205df8372d05b"
server_key_version: "encryptionkey@0"
I20260806 15:06:54.457192 28961 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:06:54.457790 28988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:54.457963 28961 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:54.458046 28961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "de66286beadc49cd9ac6396c82cc374e"
format_stamp: "Formatted at 2026-08-06 15:06:54 on dist-test-slave-tc2s"
server_key: "c6ddc27b255aed7d40c1300f48b5cac2"
server_key_iv: "b0659bc821038efa117205df8372d05b"
server_key_version: "encryptionkey@0"
I20260806 15:06:54.458106 28961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:54.472015 28961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:54.472244 28961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:54.472361 28961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:54.472554 28961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:06:54.472888 28961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:06:54.472925 28961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:54.472950 28961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:06:54.472967 28961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:54.478277 28961 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:43407
I20260806 15:06:54.478332 29151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:43407 every 8 connection(s)
I20260806 15:06:54.478601 28961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
I20260806 15:06:54.481441 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 28961
I20260806 15:06:54.481524 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-1/wal/instance
I20260806 15:06:54.481691 945 external_mini_cluster.cc:1428] Setting key ecf7e8510f70c7576aeb1a25629fe0e8
I20260806 15:06:54.482568 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:0
--local_ip_for_outbound_sockets=127.0.236.67
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:45337
--builtin_ntp_servers=127.0.236.84:42659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 15:06:54.483284 28737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:54.479216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:45927 (local address 127.0.236.126:45337)
0806 15:06:54.479260 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:54.479262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:54.479446 (+ 184us) server_negotiation.cc:407] Connection header received
0806 15:06:54.479845 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:54.479847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:54.479856 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:54.479906 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:54.480495 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.480973 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.481955 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.482201 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.482267 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:54.483051 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:54.483052 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:54.483053 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:06:54.483057 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:54.483073 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:06:54.483078 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:54.483135 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:54.483184 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 15:06:54.483207 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 15:06:54.483453 29158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:54.479187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:45337 (local address 127.0.236.66:45927)
0806 15:06:54.479345 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:54.479373 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:54.479778 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:54.479959 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:54.479962 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:54.479970 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:06:54.480436 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:54.480440 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.481032 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.481032 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:54.481857 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:54.481860 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.482290 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.482291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:54.482443 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:54.482921 (+ 478us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:06:54.482921 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:06:54.482947 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:06:54.482969 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:06:54.483137 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:06:54.483139 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:06:54.483141 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 15:06:54.483182 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 15:06:54.483253 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
I20260806 15:06:54.484477 29155 heartbeater.cc:347] Connected to a master server at 127.0.236.126:45337
I20260806 15:06:54.484571 29155 heartbeater.cc:464] Registering TS with master...
I20260806 15:06:54.484743 29155 heartbeater.cc:511] Master 127.0.236.126:45337 requested a full tablet report, sending...
I20260806 15:06:54.485059 28670 ts_manager.cc:194] Registered new tserver with Master: de66286beadc49cd9ac6396c82cc374e (127.0.236.66:43407)
I20260806 15:06:54.485419 28670 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.236.66:45927
W20260806 15:06:54.560946 29159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:54.561121 29159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:54.561141 29159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:54.562268 29159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:54.562304 29159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:54.562327 29159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:06:54.563597 29159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:42659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:45337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:54.563760 29159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:54.563984 29159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:54.568364 29174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:54.568382 29171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:54.568365 29170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:54.568593 29159 server_base.cc:1034] running on GCE node
I20260806 15:06:54.568796 29159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:54.569013 29159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:54.570150 29159 hybrid_clock.cc:648] HybridClock initialized: now 1786028814570136 us; error 30 us; skew 500 ppm
I20260806 15:06:54.571269 29159 webserver.cc:460] Webserver started at http://127.0.236.67:36261/ using document root <none> and password file <none>
I20260806 15:06:54.571483 29159 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:54.571528 29159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:54.571650 29159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:06:54.572656 29159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-2/data/instance:
uuid: "8331cef883b5457c8a0fa28cea095441"
format_stamp: "Formatted at 2026-08-06 15:06:54 on dist-test-slave-tc2s"
server_key: "fc08d8c2c1a5084b8dfd3bd96d252b62"
server_key_iv: "43e0b4c6e304f9b137263d37448a9304"
server_key_version: "encryptionkey@0"
I20260806 15:06:54.572995 29159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-2/wal/instance:
uuid: "8331cef883b5457c8a0fa28cea095441"
format_stamp: "Formatted at 2026-08-06 15:06:54 on dist-test-slave-tc2s"
server_key: "fc08d8c2c1a5084b8dfd3bd96d252b62"
server_key_iv: "43e0b4c6e304f9b137263d37448a9304"
server_key_version: "encryptionkey@0"
I20260806 15:06:54.574251 29159 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:06:54.574890 29183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:54.575035 29159 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:06:54.575109 29159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "8331cef883b5457c8a0fa28cea095441"
format_stamp: "Formatted at 2026-08-06 15:06:54 on dist-test-slave-tc2s"
server_key: "fc08d8c2c1a5084b8dfd3bd96d252b62"
server_key_iv: "43e0b4c6e304f9b137263d37448a9304"
server_key_version: "encryptionkey@0"
I20260806 15:06:54.575174 29159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:54.591637 29159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:54.591871 29159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:54.591984 29159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:54.592172 29159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:06:54.592459 29159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:06:54.592492 29159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:54.592512 29159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:06:54.592530 29159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:54.597498 29159 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:37771
I20260806 15:06:54.597528 29350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:37771 every 8 connection(s)
I20260806 15:06:54.597811 29159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
I20260806 15:06:54.602207 28737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:54.598451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:48795 (local address 127.0.236.126:45337)
0806 15:06:54.598505 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:54.598507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:54.598722 (+ 215us) server_negotiation.cc:407] Connection header received
0806 15:06:54.599147 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:54.599148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:54.599159 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:54.599207 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:54.599815 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.600285 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.601164 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.601334 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.601375 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:54.602019 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:54.602020 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:54.602020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:06:54.602024 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:54.602038 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:06:54.602041 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:54.602078 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:54.602129 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 15:06:54.602150 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 15:06:54.602375 29357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:54.598441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:45337 (local address 127.0.236.67:48795)
0806 15:06:54.598605 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:54.598647 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:54.599072 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:54.599255 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:54.599258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:54.599265 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:06:54.599753 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:54.599758 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.600355 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.600355 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:54.601105 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:54.601107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.601384 (+ 277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.601385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:54.601491 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:54.601885 (+ 394us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:06:54.601885 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:06:54.601926 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:06:54.601948 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:06:54.602079 (+ 131us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:06:54.602082 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:06:54.602084 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 15:06:54.602124 (+ 40us) client_negotiation.cc:240] Negotiation successful
0806 15:06:54.602193 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":73,"threads_started":1}
I20260806 15:06:54.602818 29352 heartbeater.cc:347] Connected to a master server at 127.0.236.126:45337
I20260806 15:06:54.602897 29352 heartbeater.cc:464] Registering TS with master...
I20260806 15:06:54.603070 29352 heartbeater.cc:511] Master 127.0.236.126:45337 requested a full tablet report, sending...
I20260806 15:06:54.603390 28670 ts_manager.cc:194] Registered new tserver with Master: 8331cef883b5457c8a0fa28cea095441 (127.0.236.67:37771)
I20260806 15:06:54.603734 28670 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.236.67:48795
I20260806 15:06:54.607734 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 29159
I20260806 15:06:54.607820 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-2/wal/instance
I20260806 15:06:54.607955 945 external_mini_cluster.cc:1428] Setting key d622f2e8eb8f2261a7d711f3470f0148
I20260806 15:06:54.608775 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:06:54.612742 28737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:54.610055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55644 (local address 127.0.236.126:45337)
0806 15:06:54.610100 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:54.610102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:54.610198 (+ 96us) server_negotiation.cc:407] Connection header received
0806 15:06:54.610259 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:54.610260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:54.610269 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:54.610315 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:54.610470 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.610927 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.611663 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.611849 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.611900 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:54.612490 (+ 590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:54.612492 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:54.612492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:06:54.612496 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:54.612509 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:06:54.612514 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:54.612583 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:54.612653 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 15:06:54.612663 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 15:06:54.614204 28670 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55644:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:06:54.615684 28670 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:06:54.621295 29374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:54.617788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:37771 (local address 127.0.0.1:56784)
0806 15:06:54.618194 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:54.618203 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:54.618257 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:54.618481 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:54.618482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:54.618484 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:54.618521 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:54.618522 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.619562 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.619563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:54.620292 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:54.620294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.621031 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.621032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:54.621109 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:54.621124 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 15:06:54.621184 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 15:06:54.621230 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":58,"threads_started":1}
I20260806 15:06:54.621305 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:54.617992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56784 (local address 127.0.236.67:37771)
0806 15:06:54.618141 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:54.618145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:54.618306 (+ 161us) server_negotiation.cc:407] Connection header received
0806 15:06:54.618341 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:54.618343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:54.618356 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:54.618417 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:54.618586 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.619494 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.620395 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.620952 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.621013 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:54.621095 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:54.621193 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 15:06:54.621228 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":71,"threads_started":1}
I20260806 15:06:54.621683 29367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:54.617433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:43407 (local address 127.0.0.1:33928)
0806 15:06:54.617546 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:54.617572 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:54.617627 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:54.617937 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:54.617940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:54.617942 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:54.618005 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:54.618006 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.619247 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.619247 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:54.620448 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:54.620451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.621425 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.621427 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:54.621517 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:54.621520 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:06:54.621566 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 15:06:54.621600 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":30,"thread_start_us":56,"threads_started":1}
I20260806 15:06:54.621769 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:54.617742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39614 (local address 127.0.236.65:45555)
0806 15:06:54.617951 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:54.617955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:54.617964 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:06:54.618004 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:54.618006 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:54.618021 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:54.618091 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:54.618253 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.619079 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.620547 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.621242 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.621308 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:54.621387 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:54.621657 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 15:06:54.621693 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":84,"threads_started":1}
I20260806 15:06:54.621850 29372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:54.617636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:45555 (local address 127.0.0.1:39614)
0806 15:06:54.617770 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:54.617779 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:54.617824 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:54.618146 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:54.618147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:54.618149 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:54.618185 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:54.618186 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.619150 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.619151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:54.620464 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:54.620466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.621431 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.621432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:54.621505 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:54.621509 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:06:54.621556 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 15:06:54.621594 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":28,"thread_start_us":70,"threads_started":1}
I20260806 15:06:54.622226 29244 tablet_service.cc:1467] Processing CreateTablet for tablet 4ac5057500764ed08e013c75031b3c4a (DEFAULT_TABLE table=test-table [id=f7ad9cbdf5ad49268eaac1905176a923]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:06:54.622527 29244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ac5057500764ed08e013c75031b3c4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:54.622429 28857 tablet_service.cc:1467] Processing CreateTablet for tablet 4ac5057500764ed08e013c75031b3c4a (DEFAULT_TABLE table=test-table [id=f7ad9cbdf5ad49268eaac1905176a923]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:06:54.622722 28857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ac5057500764ed08e013c75031b3c4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:54.624950 29377 tablet_bootstrap.cc:492] T 4ac5057500764ed08e013c75031b3c4a P 8331cef883b5457c8a0fa28cea095441: Bootstrap starting.
I20260806 15:06:54.624972 29378 tablet_bootstrap.cc:492] T 4ac5057500764ed08e013c75031b3c4a P 03def777f96149f8b5575e9e61b4262f: Bootstrap starting.
I20260806 15:06:54.625221 29368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:54.617555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33928 (local address 127.0.236.66:43407)
0806 15:06:54.617755 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:54.617760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:54.617770 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:06:54.617808 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:54.617811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:54.617827 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:54.617920 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:54.618065 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.619170 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.620547 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.621242 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.621308 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:54.621387 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:54.621602 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 15:06:54.621645 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":84,"threads_started":1}
I20260806 15:06:54.625552 29378 tablet_bootstrap.cc:654] T 4ac5057500764ed08e013c75031b3c4a P 03def777f96149f8b5575e9e61b4262f: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:54.625662 29377 tablet_bootstrap.cc:654] T 4ac5057500764ed08e013c75031b3c4a P 8331cef883b5457c8a0fa28cea095441: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:54.625845 29378 log.cc:824] T 4ac5057500764ed08e013c75031b3c4a P 03def777f96149f8b5575e9e61b4262f: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:54.625838 29051 tablet_service.cc:1467] Processing CreateTablet for tablet 4ac5057500764ed08e013c75031b3c4a (DEFAULT_TABLE table=test-table [id=f7ad9cbdf5ad49268eaac1905176a923]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:06:54.625986 29377 log.cc:824] T 4ac5057500764ed08e013c75031b3c4a P 8331cef883b5457c8a0fa28cea095441: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:54.626194 29051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ac5057500764ed08e013c75031b3c4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:54.626577 29378 tablet_bootstrap.cc:492] T 4ac5057500764ed08e013c75031b3c4a P 03def777f96149f8b5575e9e61b4262f: No bootstrap required, opened a new log
I20260806 15:06:54.626628 29377 tablet_bootstrap.cc:492] T 4ac5057500764ed08e013c75031b3c4a P 8331cef883b5457c8a0fa28cea095441: No bootstrap required, opened a new log
I20260806 15:06:54.626650 29378 ts_tablet_manager.cc:1397] T 4ac5057500764ed08e013c75031b3c4a P 03def777f96149f8b5575e9e61b4262f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:06:54.626682 29377 ts_tablet_manager.cc:1397] T 4ac5057500764ed08e013c75031b3c4a P 8331cef883b5457c8a0fa28cea095441: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:06:54.628417 29377 raft_consensus.cc:348] T 4ac5057500764ed08e013c75031b3c4a P 8331cef883b5457c8a0fa28cea095441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de66286beadc49cd9ac6396c82cc374e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 43407 } } peers { permanent_uuid: "03def777f96149f8b5575e9e61b4262f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45555 } } peers { permanent_uuid: "8331cef883b5457c8a0fa28cea095441" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37771 } }
I20260806 15:06:54.628546 29377 raft_consensus.cc:374] T 4ac5057500764ed08e013c75031b3c4a P 8331cef883b5457c8a0fa28cea095441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:54.628578 29377 raft_consensus.cc:729] T 4ac5057500764ed08e013c75031b3c4a P 8331cef883b5457c8a0fa28cea095441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8331cef883b5457c8a0fa28cea095441, State: Initialized, Role: FOLLOWER
I20260806 15:06:54.628707 29377 consensus_queue.cc:260] T 4ac5057500764ed08e013c75031b3c4a P 8331cef883b5457c8a0fa28cea095441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de66286beadc49cd9ac6396c82cc374e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 43407 } } peers { permanent_uuid: "03def777f96149f8b5575e9e61b4262f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45555 } } peers { permanent_uuid: "8331cef883b5457c8a0fa28cea095441" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37771 } }
I20260806 15:06:54.628949 29377 ts_tablet_manager.cc:1428] T 4ac5057500764ed08e013c75031b3c4a P 8331cef883b5457c8a0fa28cea095441: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:06:54.629073 29352 heartbeater.cc:503] Master 127.0.236.126:45337 was elected leader, sending a full tablet report...
I20260806 15:06:54.629017 29378 raft_consensus.cc:348] T 4ac5057500764ed08e013c75031b3c4a P 03def777f96149f8b5575e9e61b4262f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de66286beadc49cd9ac6396c82cc374e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 43407 } } peers { permanent_uuid: "03def777f96149f8b5575e9e61b4262f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45555 } } peers { permanent_uuid: "8331cef883b5457c8a0fa28cea095441" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37771 } }
I20260806 15:06:54.629156 29378 raft_consensus.cc:374] T 4ac5057500764ed08e013c75031b3c4a P 03def777f96149f8b5575e9e61b4262f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:54.629200 29378 raft_consensus.cc:729] T 4ac5057500764ed08e013c75031b3c4a P 03def777f96149f8b5575e9e61b4262f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03def777f96149f8b5575e9e61b4262f, State: Initialized, Role: FOLLOWER
I20260806 15:06:54.629340 29378 consensus_queue.cc:260] T 4ac5057500764ed08e013c75031b3c4a P 03def777f96149f8b5575e9e61b4262f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de66286beadc49cd9ac6396c82cc374e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 43407 } } peers { permanent_uuid: "03def777f96149f8b5575e9e61b4262f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45555 } } peers { permanent_uuid: "8331cef883b5457c8a0fa28cea095441" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37771 } }
I20260806 15:06:54.629594 29378 ts_tablet_manager.cc:1428] T 4ac5057500764ed08e013c75031b3c4a P 03def777f96149f8b5575e9e61b4262f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 15:06:54.629732 28958 heartbeater.cc:503] Master 127.0.236.126:45337 was elected leader, sending a full tablet report...
I20260806 15:06:54.629798 29382 tablet_bootstrap.cc:492] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e: Bootstrap starting.
I20260806 15:06:54.630424 29382 tablet_bootstrap.cc:654] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:54.630739 29382 log.cc:824] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:54.631271 29382 tablet_bootstrap.cc:492] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e: No bootstrap required, opened a new log
I20260806 15:06:54.631320 29382 ts_tablet_manager.cc:1397] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:06:54.632910 29382 raft_consensus.cc:348] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de66286beadc49cd9ac6396c82cc374e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 43407 } } peers { permanent_uuid: "03def777f96149f8b5575e9e61b4262f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45555 } } peers { permanent_uuid: "8331cef883b5457c8a0fa28cea095441" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37771 } }
I20260806 15:06:54.633030 29382 raft_consensus.cc:374] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:54.633059 29382 raft_consensus.cc:729] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de66286beadc49cd9ac6396c82cc374e, State: Initialized, Role: FOLLOWER
I20260806 15:06:54.633157 29382 consensus_queue.cc:260] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de66286beadc49cd9ac6396c82cc374e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 43407 } } peers { permanent_uuid: "03def777f96149f8b5575e9e61b4262f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45555 } } peers { permanent_uuid: "8331cef883b5457c8a0fa28cea095441" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37771 } }
I20260806 15:06:54.633399 29382 ts_tablet_manager.cc:1428] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:06:54.633447 29155 heartbeater.cc:503] Master 127.0.236.126:45337 was elected leader, sending a full tablet report...
W20260806 15:06:54.729074 29156 tablet.cc:2369] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 15:06:54.848326 29354 tablet.cc:2369] T 4ac5057500764ed08e013c75031b3c4a P 8331cef883b5457c8a0fa28cea095441: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 15:06:54.861007 28959 tablet.cc:2369] T 4ac5057500764ed08e013c75031b3c4a P 03def777f96149f8b5575e9e61b4262f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:06:54.867976 29387 raft_consensus.cc:482] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:06:54.868069 29387 raft_consensus.cc:504] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de66286beadc49cd9ac6396c82cc374e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 43407 } } peers { permanent_uuid: "03def777f96149f8b5575e9e61b4262f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45555 } } peers { permanent_uuid: "8331cef883b5457c8a0fa28cea095441" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37771 } }
I20260806 15:06:54.868353 29387 leader_election.cc:288] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03def777f96149f8b5575e9e61b4262f (127.0.236.65:45555), 8331cef883b5457c8a0fa28cea095441 (127.0.236.67:37771)
I20260806 15:06:54.872046 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:54.868710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:47509 (local address 127.0.236.65:45555)
0806 15:06:54.868754 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:54.868756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:54.868763 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:06:54.868787 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:54.868788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:54.868795 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:54.868844 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:54.869002 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.869749 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.871167 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.871710 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.871780 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:54.871790 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:54.871940 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 15:06:54.871977 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 15:06:54.872046 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:54.868710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:45129 (local address 127.0.236.67:37771)
0806 15:06:54.868754 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:54.868756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:54.868830 (+ 74us) server_negotiation.cc:407] Connection header received
0806 15:06:54.868884 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:54.868885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:54.868892 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:54.868943 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:54.869047 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.869743 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.871166 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.871727 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.871786 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:54.871792 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:54.871947 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 15:06:54.871984 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 15:06:54.872057 29158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:54.868524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:45555 (local address 127.0.236.66:47509)
0806 15:06:54.868573 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:54.868585 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:54.868629 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:54.868861 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:54.868862 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:54.868864 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:54.868933 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:54.868935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.869813 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.869814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:54.871090 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:54.871093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.871791 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.871792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:54.871870 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:54.871875 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 15:06:54.871927 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 15:06:54.871969 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":52}
I20260806 15:06:54.872063 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:54.868619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:37771 (local address 127.0.236.66:45129)
0806 15:06:54.868771 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:54.868780 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:54.868832 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:54.868938 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:54.868939 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:54.868940 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:54.868993 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:54.868994 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.869811 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.869811 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:54.871095 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:54.871096 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.871807 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.871808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:54.871877 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:54.871879 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 15:06:54.871936 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 15:06:54.871966 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":34,"thread_start_us":97,"threads_started":1}
I20260806 15:06:54.872393 28888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ac5057500764ed08e013c75031b3c4a" candidate_uuid: "de66286beadc49cd9ac6396c82cc374e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03def777f96149f8b5575e9e61b4262f" is_pre_election: true
I20260806 15:06:54.872536 28888 raft_consensus.cc:2454] T 4ac5057500764ed08e013c75031b3c4a P 03def777f96149f8b5575e9e61b4262f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de66286beadc49cd9ac6396c82cc374e in term 0.
I20260806 15:06:54.872751 28990 leader_election.cc:302] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03def777f96149f8b5575e9e61b4262f, de66286beadc49cd9ac6396c82cc374e; no voters:
I20260806 15:06:54.872900 29387 raft_consensus.cc:2781] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:06:54.872920 29275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ac5057500764ed08e013c75031b3c4a" candidate_uuid: "de66286beadc49cd9ac6396c82cc374e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8331cef883b5457c8a0fa28cea095441" is_pre_election: true
I20260806 15:06:54.872977 29387 raft_consensus.cc:482] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:06:54.873003 29387 raft_consensus.cc:3037] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:54.873034 29275 raft_consensus.cc:2454] T 4ac5057500764ed08e013c75031b3c4a P 8331cef883b5457c8a0fa28cea095441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de66286beadc49cd9ac6396c82cc374e in term 0.
I20260806 15:06:54.873752 29387 raft_consensus.cc:504] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de66286beadc49cd9ac6396c82cc374e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 43407 } } peers { permanent_uuid: "03def777f96149f8b5575e9e61b4262f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45555 } } peers { permanent_uuid: "8331cef883b5457c8a0fa28cea095441" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37771 } }
I20260806 15:06:54.873865 29387 leader_election.cc:288] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e [CANDIDATE]: Term 1 election: Requested vote from peers 03def777f96149f8b5575e9e61b4262f (127.0.236.65:45555), 8331cef883b5457c8a0fa28cea095441 (127.0.236.67:37771)
I20260806 15:06:54.874042 28888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ac5057500764ed08e013c75031b3c4a" candidate_uuid: "de66286beadc49cd9ac6396c82cc374e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03def777f96149f8b5575e9e61b4262f"
I20260806 15:06:54.874083 29275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ac5057500764ed08e013c75031b3c4a" candidate_uuid: "de66286beadc49cd9ac6396c82cc374e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8331cef883b5457c8a0fa28cea095441"
I20260806 15:06:54.874119 28888 raft_consensus.cc:3037] T 4ac5057500764ed08e013c75031b3c4a P 03def777f96149f8b5575e9e61b4262f [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:54.874150 29275 raft_consensus.cc:3037] T 4ac5057500764ed08e013c75031b3c4a P 8331cef883b5457c8a0fa28cea095441 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:54.874781 28888 raft_consensus.cc:2454] T 4ac5057500764ed08e013c75031b3c4a P 03def777f96149f8b5575e9e61b4262f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de66286beadc49cd9ac6396c82cc374e in term 1.
I20260806 15:06:54.874815 29275 raft_consensus.cc:2454] T 4ac5057500764ed08e013c75031b3c4a P 8331cef883b5457c8a0fa28cea095441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de66286beadc49cd9ac6396c82cc374e in term 1.
I20260806 15:06:54.874938 28990 leader_election.cc:302] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03def777f96149f8b5575e9e61b4262f, de66286beadc49cd9ac6396c82cc374e; no voters:
I20260806 15:06:54.875020 29387 raft_consensus.cc:2781] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:06:54.875142 29387 raft_consensus.cc:686] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e [term 1 LEADER]: Becoming Leader. State: Replica: de66286beadc49cd9ac6396c82cc374e, State: Running, Role: LEADER
I20260806 15:06:54.875234 29387 consensus_queue.cc:237] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de66286beadc49cd9ac6396c82cc374e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 43407 } } peers { permanent_uuid: "03def777f96149f8b5575e9e61b4262f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45555 } } peers { permanent_uuid: "8331cef883b5457c8a0fa28cea095441" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37771 } }
I20260806 15:06:54.875963 28670 catalog_manager.cc:5495] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e reported cstate change: term changed from 0 to 1, leader changed from <none> to de66286beadc49cd9ac6396c82cc374e (127.0.236.66). New cstate: current_term: 1 leader_uuid: "de66286beadc49cd9ac6396c82cc374e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de66286beadc49cd9ac6396c82cc374e" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 43407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03def777f96149f8b5575e9e61b4262f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8331cef883b5457c8a0fa28cea095441" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37771 } health_report { overall_health: UNKNOWN } } }
I20260806 15:06:54.900231 29368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:54.898351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33930 (local address 127.0.236.66:43407)
0806 15:06:54.898396 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:54.898397 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:54.898425 (+ 28us) server_negotiation.cc:407] Connection header received
0806 15:06:54.898452 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:54.898453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:54.898468 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:54.898516 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:06:54.898607 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.899037 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.899685 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:54.899851 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:54.899889 (+ 38us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:54.899983 (+ 94us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:06:54.900062 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:06:54.900124 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:54.900159 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 15:06:54.900166 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 15:06:54.901748 29275 raft_consensus.cc:1261] T 4ac5057500764ed08e013c75031b3c4a P 8331cef883b5457c8a0fa28cea095441 [term 1 FOLLOWER]: Refusing update from remote peer de66286beadc49cd9ac6396c82cc374e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:06:54.901770 28888 raft_consensus.cc:1261] T 4ac5057500764ed08e013c75031b3c4a P 03def777f96149f8b5575e9e61b4262f [term 1 FOLLOWER]: Refusing update from remote peer de66286beadc49cd9ac6396c82cc374e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:06:54.902088 29387 consensus_queue.cc:1035] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e [LEADER]: Connected to new peer: Peer: permanent_uuid: "03def777f96149f8b5575e9e61b4262f" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 45555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:06:54.902217 29390 consensus_queue.cc:1035] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8331cef883b5457c8a0fa28cea095441" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:06:54.904834 29395 mvcc.cc:205] Tried to move back new op lower bound from 7315574025835315200 to 7315574025729396736. Current Snapshot: MvccSnapshot[applied={T|T < 7315574025835315200}]
I20260806 15:06:54.904985 29393 mvcc.cc:205] Tried to move back new op lower bound from 7315574025835315200 to 7315574025729396736. Current Snapshot: MvccSnapshot[applied={T|T < 7315574025835315200 or (T in {7315574025835315200})}]
I20260806 15:06:54.905109 29396 mvcc.cc:205] Tried to move back new op lower bound from 7315574025835315200 to 7315574025729396736. Current Snapshot: MvccSnapshot[applied={T|T < 7315574025835315200}]
I20260806 15:06:54.906620 28666 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:06:54.906716 28666 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:06:54.907438 28666 catalog_manager.cc:5782] T 00000000000000000000000000000000 P db13968d82754af89ccb6d5333137cda: Sending DeleteTablet for 3 replicas of tablet 4ac5057500764ed08e013c75031b3c4a
I20260806 15:06:54.907820 29051 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ac5057500764ed08e013c75031b3c4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:06:54 UTC) from {username='slave'} at 127.0.0.1:33928
I20260806 15:06:54.907857 29244 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ac5057500764ed08e013c75031b3c4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:06:54 UTC) from {username='slave'} at 127.0.0.1:56784
I20260806 15:06:54.907845 28857 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ac5057500764ed08e013c75031b3c4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:06:54 UTC) from {username='slave'} at 127.0.0.1:39614
I20260806 15:06:54.908073 29411 tablet_replica.cc:330] T 4ac5057500764ed08e013c75031b3c4a P de66286beadc49cd9ac6396c82cc374e: stopping tablet replica
I20260806 15:06:54.908067 29413 tablet_replica.cc:330] T 4ac5057500764ed08e013c75031b3c4a P 03def777f96149f8b5575e9e61b4262f: stopping tablet replica
I20260806 15:06:54.908195 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 28766
W20260806 15:06:54.923332 28640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.236.65:45555 (error 108)
I20260806 15:06:54.923439 29412 tablet_replica.cc:330] T 4ac5057500764ed08e013c75031b3c4a P 8331cef883b5457c8a0fa28cea095441: stopping tablet replica
I20260806 15:06:54.923574 29412 raft_consensus.cc:2229] T 4ac5057500764ed08e013c75031b3c4a P 8331cef883b5457c8a0fa28cea095441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:06:54.923691 29412 raft_consensus.cc:2258] T 4ac5057500764ed08e013c75031b3c4a P 8331cef883b5457c8a0fa28cea095441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:06:54.923750 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 28961
I20260806 15:06:54.923960 29367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:54.923753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:45555 (local address 127.0.0.1:39616)
0806 15:06:54.923870 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:54.923924 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.65:45555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260806 15:06:54.926033 28640 catalog_manager.cc:4584] TS 03def777f96149f8b5575e9e61b4262f (127.0.236.65:45555): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4ac5057500764ed08e013c75031b3c4a: Network error: Client connection negotiation failed: client connection to 127.0.236.65:45555: connect: Connection refused (error 111)
I20260806 15:06:54.927310 29412 ts_tablet_manager.cc:1905] T 4ac5057500764ed08e013c75031b3c4a P 8331cef883b5457c8a0fa28cea095441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:06:54.929265 29412 ts_tablet_manager.cc:1918] T 4ac5057500764ed08e013c75031b3c4a P 8331cef883b5457c8a0fa28cea095441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:06:54.929335 29412 log.cc:1196] T 4ac5057500764ed08e013c75031b3c4a P 8331cef883b5457c8a0fa28cea095441: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/4ac5057500764ed08e013c75031b3c4a
I20260806 15:06:54.929540 29412 ts_tablet_manager.cc:1939] T 4ac5057500764ed08e013c75031b3c4a P 8331cef883b5457c8a0fa28cea095441: Deleting consensus metadata
I20260806 15:06:54.929951 28640 catalog_manager.cc:4841] TS 8331cef883b5457c8a0fa28cea095441 (127.0.236.67:37771): tablet 4ac5057500764ed08e013c75031b3c4a (table test-table [id=f7ad9cbdf5ad49268eaac1905176a923]) successfully deleted
I20260806 15:06:54.931811 29367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:54.931703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:43407 (local address 127.0.0.1:33932)
0806 15:06:54.931750 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:54.931781 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.66:43407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 15:06:54.931799 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 29159
I20260806 15:06:54.939581 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 28617
2026-08-06T15:06:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (916 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:06:54 dist-test-slave-tc2s krb5kdc[29419](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:06:54 dist-test-slave-tc2s krb5kdc[29419](info): set up 2 sockets
Aug 06 15:06:54 dist-test-slave-tc2s krb5kdc[29419](info): commencing operation
krb5kdc: starting...
W20260806 15:06:56.054064 945 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:06:56 dist-test-slave-tc2s krb5kdc[29419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:06:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:06:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:06:56.162057 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:44911
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:46623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:44911
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:06:56.241163 29443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:56.241344 29443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:56.241364 29443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:56.242506 29443 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:06:56.242547 29443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:56.242559 29443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:56.242581 29443 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:06:56.242605 29443 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:06:56.244032 29443 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:46623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:44911
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:44911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:56.244246 29443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:56.244444 29443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:56.248838 29451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:56.248908 29457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:56.248956 29443 server_base.cc:1034] running on GCE node
W20260806 15:06:56.248852 29453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:56.249258 29443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:56.249505 29443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:56.250630 29443 hybrid_clock.cc:648] HybridClock initialized: now 1786028816250619 us; error 27 us; skew 500 ppm
Aug 06 15:06:56 dist-test-slave-tc2s krb5kdc[29419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028816, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:06:56.252912 29443 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:06:56.253301 29443 webserver.cc:460] Webserver started at http://127.0.236.126:38681/ using document root <none> and password file <none>
I20260806 15:06:56.253508 29443 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:56.253552 29443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:56.253666 29443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:06:56.254652 29443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "fdb79cddb864445993dc86b61198dcb4"
format_stamp: "Formatted at 2026-08-06 15:06:56 on dist-test-slave-tc2s"
server_key: "2d5ec9f6f9cf5d8102bbb22da6f0df9b"
server_key_iv: "50bf456e54b04796bd995ad6d05d7e74"
server_key_version: "encryptionkey@0"
I20260806 15:06:56.254963 29443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "fdb79cddb864445993dc86b61198dcb4"
format_stamp: "Formatted at 2026-08-06 15:06:56 on dist-test-slave-tc2s"
server_key: "2d5ec9f6f9cf5d8102bbb22da6f0df9b"
server_key_iv: "50bf456e54b04796bd995ad6d05d7e74"
server_key_version: "encryptionkey@0"
I20260806 15:06:56.256161 29443 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 15:06:56.256798 29467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:56.256947 29443 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:06:56.257023 29443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "fdb79cddb864445993dc86b61198dcb4"
format_stamp: "Formatted at 2026-08-06 15:06:56 on dist-test-slave-tc2s"
server_key: "2d5ec9f6f9cf5d8102bbb22da6f0df9b"
server_key_iv: "50bf456e54b04796bd995ad6d05d7e74"
server_key_version: "encryptionkey@0"
I20260806 15:06:56.257087 29443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:56.276273 29443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:56.278250 29443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:56.278383 29443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:56.282097 29443 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:44911
I20260806 15:06:56.282142 29551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:44911 every 8 connection(s)
I20260806 15:06:56.282433 29443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:06:56.283214 29553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:56.285240 29553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdb79cddb864445993dc86b61198dcb4: Bootstrap starting.
I20260806 15:06:56.285765 29553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdb79cddb864445993dc86b61198dcb4: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:56.286068 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 29443
I20260806 15:06:56.286072 29553 log.cc:824] T 00000000000000000000000000000000 P fdb79cddb864445993dc86b61198dcb4: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:56.286164 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:06:56.286341 945 external_mini_cluster.cc:1428] Setting key 0774e3dcd3e577ab289198078cdaf5b1
I20260806 15:06:56.286736 29553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdb79cddb864445993dc86b61198dcb4: No bootstrap required, opened a new log
I20260806 15:06:56.288480 29553 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdb79cddb864445993dc86b61198dcb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb79cddb864445993dc86b61198dcb4" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44911 } }
I20260806 15:06:56.288606 29553 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdb79cddb864445993dc86b61198dcb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:56.288645 29553 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdb79cddb864445993dc86b61198dcb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdb79cddb864445993dc86b61198dcb4, State: Initialized, Role: FOLLOWER
I20260806 15:06:56.288762 29553 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdb79cddb864445993dc86b61198dcb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb79cddb864445993dc86b61198dcb4" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44911 } }
I20260806 15:06:56.288834 29553 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdb79cddb864445993dc86b61198dcb4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:06:56.288864 29553 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdb79cddb864445993dc86b61198dcb4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:06:56.288910 29553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdb79cddb864445993dc86b61198dcb4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:56.289469 29553 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdb79cddb864445993dc86b61198dcb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb79cddb864445993dc86b61198dcb4" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44911 } }
I20260806 15:06:56.289563 29553 leader_election.cc:302] T 00000000000000000000000000000000 P fdb79cddb864445993dc86b61198dcb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdb79cddb864445993dc86b61198dcb4; no voters:
I20260806 15:06:56.289721 29553 leader_election.cc:288] T 00000000000000000000000000000000 P fdb79cddb864445993dc86b61198dcb4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:06:56.289757 29561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdb79cddb864445993dc86b61198dcb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:06:56.289963 29561 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdb79cddb864445993dc86b61198dcb4 [term 1 LEADER]: Becoming Leader. State: Replica: fdb79cddb864445993dc86b61198dcb4, State: Running, Role: LEADER
I20260806 15:06:56.289983 29553 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdb79cddb864445993dc86b61198dcb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:06:56.290048 29561 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdb79cddb864445993dc86b61198dcb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb79cddb864445993dc86b61198dcb4" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44911 } }
I20260806 15:06:56.290380 29561 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdb79cddb864445993dc86b61198dcb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdb79cddb864445993dc86b61198dcb4. Latest consensus state: current_term: 1 leader_uuid: "fdb79cddb864445993dc86b61198dcb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb79cddb864445993dc86b61198dcb4" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44911 } } }
I20260806 15:06:56.290454 29561 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdb79cddb864445993dc86b61198dcb4 [sys.catalog]: This master's current role is: LEADER
I20260806 15:06:56.290460 29565 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdb79cddb864445993dc86b61198dcb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdb79cddb864445993dc86b61198dcb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb79cddb864445993dc86b61198dcb4" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44911 } } }
I20260806 15:06:56.290673 29565 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdb79cddb864445993dc86b61198dcb4 [sys.catalog]: This master's current role is: LEADER
W20260806 15:06:56.291888 29589 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fdb79cddb864445993dc86b61198dcb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:06:56.291941 29589 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:06:56.292286 29574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:06:56.292510 29574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 15:06:56 dist-test-slave-tc2s krb5kdc[29419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:06:56.294131 29574 catalog_manager.cc:1347] Generated new cluster ID: e2e66d9ab7fc4f5e8af7630f6b72c380
I20260806 15:06:56.294190 29574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:06:56.297061 29559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:56.286777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60444 (local address 127.0.236.126:44911)
0806 15:06:56.286967 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:56.286977 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:56.287000 (+ 23us) server_negotiation.cc:407] Connection header received
0806 15:06:56.287849 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:56.287856 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:56.287898 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:56.288017 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:56.288612 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.290075 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.291072 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.291645 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.291749 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:56.294262 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:56.294265 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:56.294266 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:06:56.294277 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:56.295731 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:56.296046 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:56.296049 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:56.296049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:56.296129 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:56.296300 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:56.296301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:56.296301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:56.296527 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:56.296684 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:56.296778 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 15:06:56.296902 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":144,"thread_start_us":88,"threads_started":1}
I20260806 15:06:56.300091 29574 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:06:56.300769 29574 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:06:56.306015 29574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdb79cddb864445993dc86b61198dcb4: Generated new TSK 0
I20260806 15:06:56.306165 29574 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.236.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
Entry for principal kudu/127.0.236.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab.
I20260806 15:06:56.334501 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:0
--local_ip_for_outbound_sockets=127.0.236.65
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44911
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:46623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:06:56.449493 29597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:56.449749 29597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:56.449781 29597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:56.451498 29597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:56.451555 29597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:56.451591 29597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:06:56.453864 29597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:46623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/kudu_127.0.236.65.keytab
--principal=kudu/127.0.236.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:56.454115 29597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:56.454339 29597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:56.459247 29602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:56.459211 29610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:56.459405 29604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:56.459709 29597 server_base.cc:1034] running on GCE node
I20260806 15:06:56.459882 29597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:56.460100 29597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:56.461236 29597 hybrid_clock.cc:648] HybridClock initialized: now 1786028816461223 us; error 29 us; skew 500 ppm
Aug 06 15:06:56 dist-test-slave-tc2s krb5kdc[29419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028816, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:06:56.463627 29597 init.cc:377] Logged in from keytab as kudu/127.0.236.65@KRBTEST.COM (short username kudu)
I20260806 15:06:56.464018 29597 webserver.cc:460] Webserver started at http://127.0.236.65:42611/ using document root <none> and password file <none>
I20260806 15:06:56.464221 29597 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:56.464262 29597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:56.464373 29597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:06:56.465328 29597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-0/data/instance:
uuid: "f0da1bff1f7b46cdb70ce83d013b3770"
format_stamp: "Formatted at 2026-08-06 15:06:56 on dist-test-slave-tc2s"
server_key: "e70d6954917831709bd2a40407188e30"
server_key_iv: "287a5d506fda553b46f397f061d0be71"
server_key_version: "encryptionkey@0"
I20260806 15:06:56.465627 29597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-0/wal/instance:
uuid: "f0da1bff1f7b46cdb70ce83d013b3770"
format_stamp: "Formatted at 2026-08-06 15:06:56 on dist-test-slave-tc2s"
server_key: "e70d6954917831709bd2a40407188e30"
server_key_iv: "287a5d506fda553b46f397f061d0be71"
server_key_version: "encryptionkey@0"
I20260806 15:06:56.466832 29597 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:06:56.467450 29620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:56.467599 29597 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:06:56.467684 29597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "f0da1bff1f7b46cdb70ce83d013b3770"
format_stamp: "Formatted at 2026-08-06 15:06:56 on dist-test-slave-tc2s"
server_key: "e70d6954917831709bd2a40407188e30"
server_key_iv: "287a5d506fda553b46f397f061d0be71"
server_key_version: "encryptionkey@0"
I20260806 15:06:56.467736 29597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:56.480237 29597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:56.481796 29597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:56.481936 29597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:56.482115 29597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:06:56.482345 29597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:06:56.482374 29597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:56.482400 29597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:06:56.482415 29597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:56.487754 29597 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:46541
I20260806 15:06:56.487818 29806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:46541 every 8 connection(s)
I20260806 15:06:56.488081 29597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:06:56.489526 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 29597
I20260806 15:06:56.489643 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-0/wal/instance
I20260806 15:06:56.489813 945 external_mini_cluster.cc:1428] Setting key cd27437ebb521b5ab1f88e2e2d32a41a
Aug 06 15:06:56 dist-test-slave-tc2s krb5kdc[29419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028816, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.65@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:06:56.497522 29559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:56.488718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:35959 (local address 127.0.236.126:44911)
0806 15:06:56.488777 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:56.488779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:56.488953 (+ 174us) server_negotiation.cc:407] Connection header received
0806 15:06:56.489851 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:56.489853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:56.489864 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:56.489974 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:56.490727 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.491408 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.492413 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.492630 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.492687 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:56.494992 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:56.494993 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:56.494993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:06:56.495000 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:56.496216 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:56.496673 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:56.496674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:56.496674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:56.496719 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:56.496963 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:56.496963 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:56.496964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:56.497058 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:56.497102 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:56.497391 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 15:06:56.497457 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 15:06:56.497769 29811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:56.488690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:44911 (local address 127.0.236.65:35959)
0806 15:06:56.488846 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:56.488880 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:56.489753 (+ 873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:56.490040 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:56.490045 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:56.490369 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:06:56.490627 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:56.490633 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.491486 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.491487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:56.492330 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:56.492332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.492703 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.492704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:56.492830 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:56.493328 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:06:56.493336 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:06:56.494905 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:06:56.496292 (+ 1387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:56.496301 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:56.496303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:56.496608 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:56.496769 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:56.496770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:56.496770 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:56.496907 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:56.497108 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:06:56.497114 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:06:56.497272 (+ 158us) client_negotiation.cc:769] Sending connection context
0806 15:06:56.497384 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 15:06:56.497555 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
I20260806 15:06:56.498296 29807 heartbeater.cc:347] Connected to a master server at 127.0.236.126:44911
I20260806 15:06:56.498396 29807 heartbeater.cc:464] Registering TS with master...
I20260806 15:06:56.498594 29807 heartbeater.cc:511] Master 127.0.236.126:44911 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.0.236.66@KRBTEST.COM; defaulting to no policy
I20260806 15:06:56.499140 29491 ts_manager.cc:194] Registered new tserver with Master: f0da1bff1f7b46cdb70ce83d013b3770 (127.0.236.65:46541)
I20260806 15:06:56.499752 29491 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.65@KRBTEST.COM'} at 127.0.236.65:35959
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
Entry for principal kudu/127.0.236.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab.
I20260806 15:06:56.513818 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:0
--local_ip_for_outbound_sockets=127.0.236.66
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44911
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:46623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:06:56.591320 29815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:56.591506 29815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:56.591528 29815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:56.592630 29815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:56.592669 29815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:56.592691 29815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:06:56.594105 29815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:46623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/kudu_127.0.236.66.keytab
--principal=kudu/127.0.236.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:56.594267 29815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:56.594427 29815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:56.598810 29828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:56.598810 29824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:56.598822 29826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:56.599049 29815 server_base.cc:1034] running on GCE node
I20260806 15:06:56.599277 29815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:56.599472 29815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:56.600605 29815 hybrid_clock.cc:648] HybridClock initialized: now 1786028816600591 us; error 27 us; skew 500 ppm
Aug 06 15:06:56 dist-test-slave-tc2s krb5kdc[29419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028816, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:06:56.602952 29815 init.cc:377] Logged in from keytab as kudu/127.0.236.66@KRBTEST.COM (short username kudu)
I20260806 15:06:56.603340 29815 webserver.cc:460] Webserver started at http://127.0.236.66:46651/ using document root <none> and password file <none>
I20260806 15:06:56.603526 29815 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:56.603571 29815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:56.603693 29815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:06:56.604631 29815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-1/data/instance:
uuid: "d4a91efb8a1c4eac9d8861392e8bdd67"
format_stamp: "Formatted at 2026-08-06 15:06:56 on dist-test-slave-tc2s"
server_key: "830d9a60162fafceada799e246f951c6"
server_key_iv: "1a7f4da8347405bb06c031c8156342be"
server_key_version: "encryptionkey@0"
I20260806 15:06:56.604948 29815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-1/wal/instance:
uuid: "d4a91efb8a1c4eac9d8861392e8bdd67"
format_stamp: "Formatted at 2026-08-06 15:06:56 on dist-test-slave-tc2s"
server_key: "830d9a60162fafceada799e246f951c6"
server_key_iv: "1a7f4da8347405bb06c031c8156342be"
server_key_version: "encryptionkey@0"
I20260806 15:06:56.606138 29815 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:06:56.606760 29835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:56.606910 29815 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:56.607009 29815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "d4a91efb8a1c4eac9d8861392e8bdd67"
format_stamp: "Formatted at 2026-08-06 15:06:56 on dist-test-slave-tc2s"
server_key: "830d9a60162fafceada799e246f951c6"
server_key_iv: "1a7f4da8347405bb06c031c8156342be"
server_key_version: "encryptionkey@0"
I20260806 15:06:56.607074 29815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:56.641222 29815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:56.642853 29815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:56.642987 29815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:56.643179 29815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:06:56.643445 29815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:06:56.643478 29815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:56.643504 29815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:06:56.643522 29815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:56.648809 29815 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:42065
I20260806 15:06:56.648880 30000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:42065 every 8 connection(s)
I20260806 15:06:56.649127 29815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
Aug 06 15:06:56 dist-test-slave-tc2s krb5kdc[29419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028816, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.66@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:06:56.657008 29559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:56.649763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:38119 (local address 127.0.236.126:44911)
0806 15:06:56.649808 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:56.649810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:56.649994 (+ 184us) server_negotiation.cc:407] Connection header received
0806 15:06:56.650810 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:56.650812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:56.650820 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:56.650868 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:56.651385 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.651858 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.652747 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.652929 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.652970 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:56.654729 (+ 1759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:56.654731 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:56.654731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:06:56.654738 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:56.655956 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:56.656262 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:56.656263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:56.656263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:56.656308 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:56.656553 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:56.656554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:56.656554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:56.656644 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:56.656686 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:56.656882 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 15:06:56.656946 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 15:06:56.657172 30006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:56.649722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:44911 (local address 127.0.236.66:38119)
0806 15:06:56.649864 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:56.649923 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:56.650736 (+ 813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:56.650917 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:56.650920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:56.651154 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:06:56.651325 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:56.651329 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.651914 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.651915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:56.652689 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:56.652691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.652979 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.652980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:56.653068 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:56.653426 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:06:56.653433 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:06:56.654640 (+ 1207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:06:56.656014 (+ 1374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:56.656019 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:56.656021 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:56.656212 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:56.656370 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:56.656370 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:56.656370 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:56.656501 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:56.656686 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:06:56.656690 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:06:56.656794 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 15:06:56.656866 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 15:06:56.656986 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":76,"threads_started":1}
I20260806 15:06:56.657608 30001 heartbeater.cc:347] Connected to a master server at 127.0.236.126:44911
I20260806 15:06:56.657688 30001 heartbeater.cc:464] Registering TS with master...
I20260806 15:06:56.657845 30001 heartbeater.cc:511] Master 127.0.236.126:44911 requested a full tablet report, sending...
I20260806 15:06:56.658115 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 29815
I20260806 15:06:56.658232 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-1/wal/instance
I20260806 15:06:56.658233 29491 ts_manager.cc:194] Registered new tserver with Master: d4a91efb8a1c4eac9d8861392e8bdd67 (127.0.236.66:42065)
I20260806 15:06:56.658409 945 external_mini_cluster.cc:1428] Setting key a927b04a3c0585e4878db3c86cd37bec
I20260806 15:06:56.658618 29491 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.66@KRBTEST.COM'} at 127.0.236.66:38119
WARNING: no policy specified for kudu/127.0.236.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
Entry for principal kudu/127.0.236.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab.
I20260806 15:06:56.681331 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:0
--local_ip_for_outbound_sockets=127.0.236.67
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44911
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:46623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:06:56.759217 30011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:56.759416 30011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:56.759439 30011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:56.760548 30011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:56.760588 30011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:56.760615 30011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:06:56.762169 30011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:46623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/krb5kdc/kudu_127.0.236.67.keytab
--principal=kudu/127.0.236.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:44911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:56.762388 30011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:56.762588 30011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:56.767079 30018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:56.767143 30022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:56.767331 30011 server_base.cc:1034] running on GCE node
W20260806 15:06:56.767406 30020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:56.767619 30011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:56.767839 30011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:56.768970 30011 hybrid_clock.cc:648] HybridClock initialized: now 1786028816768958 us; error 25 us; skew 500 ppm
Aug 06 15:06:56 dist-test-slave-tc2s krb5kdc[29419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028816, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:06:56.771379 30011 init.cc:377] Logged in from keytab as kudu/127.0.236.67@KRBTEST.COM (short username kudu)
I20260806 15:06:56.771795 30011 webserver.cc:460] Webserver started at http://127.0.236.67:45927/ using document root <none> and password file <none>
I20260806 15:06:56.771981 30011 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:56.772027 30011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:56.772145 30011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:06:56.773056 30011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-2/data/instance:
uuid: "a458c962bc604aaeb2afeafefd781cdb"
format_stamp: "Formatted at 2026-08-06 15:06:56 on dist-test-slave-tc2s"
server_key: "4a48ef003ff2f7c8a8641a4907e673bd"
server_key_iv: "7c3a6358bc0e7350dfca78ab132d6f1e"
server_key_version: "encryptionkey@0"
I20260806 15:06:56.773356 30011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-2/wal/instance:
uuid: "a458c962bc604aaeb2afeafefd781cdb"
format_stamp: "Formatted at 2026-08-06 15:06:56 on dist-test-slave-tc2s"
server_key: "4a48ef003ff2f7c8a8641a4907e673bd"
server_key_iv: "7c3a6358bc0e7350dfca78ab132d6f1e"
server_key_version: "encryptionkey@0"
I20260806 15:06:56.774580 30011 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:06:56.775210 30030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:56.775357 30011 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:06:56.775445 30011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "a458c962bc604aaeb2afeafefd781cdb"
format_stamp: "Formatted at 2026-08-06 15:06:56 on dist-test-slave-tc2s"
server_key: "4a48ef003ff2f7c8a8641a4907e673bd"
server_key_iv: "7c3a6358bc0e7350dfca78ab132d6f1e"
server_key_version: "encryptionkey@0"
I20260806 15:06:56.775501 30011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:56.812496 30011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:56.814471 30011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:56.814599 30011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:56.814792 30011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:06:56.815057 30011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:06:56.815090 30011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:56.815117 30011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:06:56.815141 30011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:56.820305 30011 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:36115
I20260806 15:06:56.820374 30198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:36115 every 8 connection(s)
I20260806 15:06:56.820636 30011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
I20260806 15:06:56.825753 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 30011
I20260806 15:06:56.825839 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-2/wal/instance
Aug 06 15:06:56 dist-test-slave-tc2s krb5kdc[29419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028816, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.67@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:06:56.826027 945 external_mini_cluster.cc:1428] Setting key 6062c52a15d8dde2824e30632dcc5997
I20260806 15:06:56.828843 29559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:56.821258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:35253 (local address 127.0.236.126:44911)
0806 15:06:56.821307 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:56.821309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:56.821500 (+ 191us) server_negotiation.cc:407] Connection header received
0806 15:06:56.822375 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:56.822376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:56.822385 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:56.822427 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:56.822949 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.823419 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.824316 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.824489 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.824528 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:56.826389 (+ 1861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:56.826391 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:56.826391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:06:56.826399 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:56.827652 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:56.828004 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:56.828005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:56.828006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:56.828064 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:56.828271 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:56.828272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:56.828272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:56.828380 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:56.828434 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:56.828678 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 15:06:56.828754 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 15:06:56.828981 30202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:56.821223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:44911 (local address 127.0.236.67:35253)
0806 15:06:56.821386 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:56.821426 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:56.822279 (+ 853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:56.822476 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:56.822479 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:56.822717 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:06:56.822889 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:56.822893 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.823474 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.823475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:56.824257 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:56.824259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.824541 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.824542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:56.824632 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:56.824983 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:06:56.824991 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:06:56.826314 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:06:56.827726 (+ 1412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:56.827732 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:56.827734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:56.827939 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:56.828114 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:06:56.828115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:06:56.828115 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:06:56.828216 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:06:56.828437 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:06:56.828443 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:06:56.828567 (+ 124us) client_negotiation.cc:769] Sending connection context
0806 15:06:56.828649 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 15:06:56.828788 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260806 15:06:56.829324 30200 heartbeater.cc:347] Connected to a master server at 127.0.236.126:44911
I20260806 15:06:56.829406 30200 heartbeater.cc:464] Registering TS with master...
I20260806 15:06:56.829571 30200 heartbeater.cc:511] Master 127.0.236.126:44911 requested a full tablet report, sending...
I20260806 15:06:56.829877 29490 ts_manager.cc:194] Registered new tserver with Master: a458c962bc604aaeb2afeafefd781cdb (127.0.236.67:36115)
I20260806 15:06:56.830260 29490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.236.67@KRBTEST.COM'} at 127.0.236.67:35253
I20260806 15:06:56.837477 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:06:56.843824 29559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:56.838704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60450 (local address 127.0.236.126:44911)
0806 15:06:56.838756 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:56.838758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:56.838830 (+ 72us) server_negotiation.cc:407] Connection header received
0806 15:06:56.838889 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:56.838890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:56.838899 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:56.838949 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:56.839462 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.839913 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.840613 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.840794 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.840846 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:56.841828 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:56.841830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:56.841830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:06:56.841837 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:56.842827 (+ 990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:56.843130 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:56.843132 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:56.843132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:56.843184 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:56.843343 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:56.843344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:56.843344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:56.843478 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:56.843572 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:56.843672 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 15:06:56.843740 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 15:06:56.845175 29490 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60450:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:06:56.846714 29490 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:06:56.852563 30216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:56.848629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:46541 (local address 127.0.0.1:37402)
0806 15:06:56.848944 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:56.848984 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:56.849046 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:56.849304 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:56.849305 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:56.849308 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:56.849362 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:56.849364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.850617 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.850619 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:56.851492 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:56.851495 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.852273 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.852274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:56.852353 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:56.852366 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 15:06:56.852446 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 15:06:56.852494 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":32,"thread_start_us":75,"threads_started":1}
I20260806 15:06:56.852583 30222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:56.848814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37402 (local address 127.0.236.65:46541)
0806 15:06:56.849185 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:56.849188 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:56.849196 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:06:56.849225 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:56.849227 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:56.849237 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:56.849296 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:56.849427 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.850516 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.851579 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.852190 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.852262 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:56.852316 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:56.852465 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 15:06:56.852502 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":76,"threads_started":1}
I20260806 15:06:56.852926 30220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:56.848817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:36115 (local address 127.0.0.1:42348)
0806 15:06:56.849408 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:56.849417 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:56.849467 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:56.849720 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:56.849721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:56.849723 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:56.849781 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:56.849782 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.850634 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.850635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:56.851904 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:56.851907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.852712 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.852713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:56.852771 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:56.852775 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:06:56.852816 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 15:06:56.852854 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"mutex_wait_us":97,"thread_start_us":70,"threads_started":1}
I20260806 15:06:56.853142 30223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:56.848875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42348 (local address 127.0.236.67:36115)
0806 15:06:56.849045 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:56.849048 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:56.849525 (+ 477us) server_negotiation.cc:407] Connection header received
0806 15:06:56.849565 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:56.849567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:56.849580 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:56.849645 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:56.849846 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.850533 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.851997 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.852561 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.852619 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:56.852676 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:56.853036 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 15:06:56.853071 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":88,"threads_started":1}
I20260806 15:06:56.853186 30219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:56.848750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55346 (local address 127.0.236.66:42065)
0806 15:06:56.848999 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:56.849003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:56.849062 (+ 59us) server_negotiation.cc:407] Connection header received
0806 15:06:56.849129 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:56.849131 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:56.849144 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:56.849219 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:56.849364 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.850484 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.851987 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.852684 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.852753 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:56.852807 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:56.853067 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 15:06:56.853105 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":84,"threads_started":1}
I20260806 15:06:56.853192 30218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:56.848625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:42065 (local address 127.0.0.1:55346)
0806 15:06:56.849007 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:56.849016 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:56.849068 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:56.849242 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:56.849246 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:56.849248 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:56.849308 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:56.849309 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.850596 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.850597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:56.851904 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:56.851907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.852756 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.852757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:56.852832 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:56.852837 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 15:06:56.852887 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 15:06:56.852928 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":90,"thread_start_us":59,"threads_started":1}
I20260806 15:06:56.853545 29694 tablet_service.cc:1467] Processing CreateTablet for tablet aa23e369e3f24e0fadb20562a734f16d (DEFAULT_TABLE table=test-table [id=03f2bb914c824d6da0f97eec55d4086d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:06:56.853688 30103 tablet_service.cc:1467] Processing CreateTablet for tablet aa23e369e3f24e0fadb20562a734f16d (DEFAULT_TABLE table=test-table [id=03f2bb914c824d6da0f97eec55d4086d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:06:56.853848 29903 tablet_service.cc:1467] Processing CreateTablet for tablet aa23e369e3f24e0fadb20562a734f16d (DEFAULT_TABLE table=test-table [id=03f2bb914c824d6da0f97eec55d4086d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:06:56.854032 30103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa23e369e3f24e0fadb20562a734f16d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:56.854194 29903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa23e369e3f24e0fadb20562a734f16d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:56.853873 29694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa23e369e3f24e0fadb20562a734f16d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:56.856398 30224 tablet_bootstrap.cc:492] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb: Bootstrap starting.
I20260806 15:06:56.857285 30226 tablet_bootstrap.cc:492] T aa23e369e3f24e0fadb20562a734f16d P d4a91efb8a1c4eac9d8861392e8bdd67: Bootstrap starting.
I20260806 15:06:56.857400 30224 tablet_bootstrap.cc:654] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:56.857954 30226 tablet_bootstrap.cc:654] T aa23e369e3f24e0fadb20562a734f16d P d4a91efb8a1c4eac9d8861392e8bdd67: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:56.858022 30224 log.cc:824] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:56.858039 30225 tablet_bootstrap.cc:492] T aa23e369e3f24e0fadb20562a734f16d P f0da1bff1f7b46cdb70ce83d013b3770: Bootstrap starting.
I20260806 15:06:56.858453 30226 log.cc:824] T aa23e369e3f24e0fadb20562a734f16d P d4a91efb8a1c4eac9d8861392e8bdd67: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:56.858631 30225 tablet_bootstrap.cc:654] T aa23e369e3f24e0fadb20562a734f16d P f0da1bff1f7b46cdb70ce83d013b3770: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:56.858824 30224 tablet_bootstrap.cc:492] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb: No bootstrap required, opened a new log
I20260806 15:06:56.858884 30224 ts_tablet_manager.cc:1397] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 15:06:56.859172 30225 log.cc:824] T aa23e369e3f24e0fadb20562a734f16d P f0da1bff1f7b46cdb70ce83d013b3770: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:56.859196 30226 tablet_bootstrap.cc:492] T aa23e369e3f24e0fadb20562a734f16d P d4a91efb8a1c4eac9d8861392e8bdd67: No bootstrap required, opened a new log
I20260806 15:06:56.859259 30226 ts_tablet_manager.cc:1397] T aa23e369e3f24e0fadb20562a734f16d P d4a91efb8a1c4eac9d8861392e8bdd67: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 15:06:56.859905 30225 tablet_bootstrap.cc:492] T aa23e369e3f24e0fadb20562a734f16d P f0da1bff1f7b46cdb70ce83d013b3770: No bootstrap required, opened a new log
I20260806 15:06:56.859982 30225 ts_tablet_manager.cc:1397] T aa23e369e3f24e0fadb20562a734f16d P f0da1bff1f7b46cdb70ce83d013b3770: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:06:56.860669 30224 raft_consensus.cc:348] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4a91efb8a1c4eac9d8861392e8bdd67" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42065 } } peers { permanent_uuid: "f0da1bff1f7b46cdb70ce83d013b3770" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46541 } } peers { permanent_uuid: "a458c962bc604aaeb2afeafefd781cdb" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 36115 } }
I20260806 15:06:56.860802 30224 raft_consensus.cc:374] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:56.860836 30224 raft_consensus.cc:729] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a458c962bc604aaeb2afeafefd781cdb, State: Initialized, Role: FOLLOWER
I20260806 15:06:56.860963 30224 consensus_queue.cc:260] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4a91efb8a1c4eac9d8861392e8bdd67" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42065 } } peers { permanent_uuid: "f0da1bff1f7b46cdb70ce83d013b3770" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46541 } } peers { permanent_uuid: "a458c962bc604aaeb2afeafefd781cdb" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 36115 } }
I20260806 15:06:56.861208 30224 ts_tablet_manager.cc:1428] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 15:06:56.861090 30226 raft_consensus.cc:348] T aa23e369e3f24e0fadb20562a734f16d P d4a91efb8a1c4eac9d8861392e8bdd67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4a91efb8a1c4eac9d8861392e8bdd67" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42065 } } peers { permanent_uuid: "f0da1bff1f7b46cdb70ce83d013b3770" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46541 } } peers { permanent_uuid: "a458c962bc604aaeb2afeafefd781cdb" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 36115 } }
I20260806 15:06:56.861246 30226 raft_consensus.cc:374] T aa23e369e3f24e0fadb20562a734f16d P d4a91efb8a1c4eac9d8861392e8bdd67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:56.861289 30226 raft_consensus.cc:729] T aa23e369e3f24e0fadb20562a734f16d P d4a91efb8a1c4eac9d8861392e8bdd67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4a91efb8a1c4eac9d8861392e8bdd67, State: Initialized, Role: FOLLOWER
I20260806 15:06:56.861311 30200 heartbeater.cc:503] Master 127.0.236.126:44911 was elected leader, sending a full tablet report...
I20260806 15:06:56.861400 30226 consensus_queue.cc:260] T aa23e369e3f24e0fadb20562a734f16d P d4a91efb8a1c4eac9d8861392e8bdd67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4a91efb8a1c4eac9d8861392e8bdd67" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42065 } } peers { permanent_uuid: "f0da1bff1f7b46cdb70ce83d013b3770" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46541 } } peers { permanent_uuid: "a458c962bc604aaeb2afeafefd781cdb" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 36115 } }
I20260806 15:06:56.861666 30226 ts_tablet_manager.cc:1428] T aa23e369e3f24e0fadb20562a734f16d P d4a91efb8a1c4eac9d8861392e8bdd67: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 15:06:56.861708 30001 heartbeater.cc:503] Master 127.0.236.126:44911 was elected leader, sending a full tablet report...
I20260806 15:06:56.862082 30225 raft_consensus.cc:348] T aa23e369e3f24e0fadb20562a734f16d P f0da1bff1f7b46cdb70ce83d013b3770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4a91efb8a1c4eac9d8861392e8bdd67" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42065 } } peers { permanent_uuid: "f0da1bff1f7b46cdb70ce83d013b3770" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46541 } } peers { permanent_uuid: "a458c962bc604aaeb2afeafefd781cdb" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 36115 } }
I20260806 15:06:56.862214 30225 raft_consensus.cc:374] T aa23e369e3f24e0fadb20562a734f16d P f0da1bff1f7b46cdb70ce83d013b3770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:56.862246 30225 raft_consensus.cc:729] T aa23e369e3f24e0fadb20562a734f16d P f0da1bff1f7b46cdb70ce83d013b3770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0da1bff1f7b46cdb70ce83d013b3770, State: Initialized, Role: FOLLOWER
I20260806 15:06:56.862352 30225 consensus_queue.cc:260] T aa23e369e3f24e0fadb20562a734f16d P f0da1bff1f7b46cdb70ce83d013b3770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4a91efb8a1c4eac9d8861392e8bdd67" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42065 } } peers { permanent_uuid: "f0da1bff1f7b46cdb70ce83d013b3770" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46541 } } peers { permanent_uuid: "a458c962bc604aaeb2afeafefd781cdb" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 36115 } }
I20260806 15:06:56.862615 30225 ts_tablet_manager.cc:1428] T aa23e369e3f24e0fadb20562a734f16d P f0da1bff1f7b46cdb70ce83d013b3770: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 15:06:56.862735 29807 heartbeater.cc:503] Master 127.0.236.126:44911 was elected leader, sending a full tablet report...
W20260806 15:06:56.899613 30004 tablet.cc:2369] T aa23e369e3f24e0fadb20562a734f16d P d4a91efb8a1c4eac9d8861392e8bdd67: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:06:56.914323 30230 raft_consensus.cc:482] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:06:56.914417 30230 raft_consensus.cc:504] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4a91efb8a1c4eac9d8861392e8bdd67" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42065 } } peers { permanent_uuid: "f0da1bff1f7b46cdb70ce83d013b3770" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46541 } } peers { permanent_uuid: "a458c962bc604aaeb2afeafefd781cdb" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 36115 } }
I20260806 15:06:56.914690 30230 leader_election.cc:288] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4a91efb8a1c4eac9d8861392e8bdd67 (127.0.236.66:42065), f0da1bff1f7b46cdb70ce83d013b3770 (127.0.236.65:46541)
I20260806 15:06:56.918267 30219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:56.914929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:34567 (local address 127.0.236.66:42065)
0806 15:06:56.914984 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:56.914986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:56.914993 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:06:56.915018 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:56.915018 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:56.915029 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:56.915085 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:56.915224 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.915956 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.917328 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.917923 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.917995 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:56.918004 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:56.918165 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 15:06:56.918199 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 15:06:56.918267 30222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:56.914907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:57971 (local address 127.0.236.65:46541)
0806 15:06:56.914971 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:56.914973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:56.915100 (+ 127us) server_negotiation.cc:407] Connection header received
0806 15:06:56.915127 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:56.915128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:56.915138 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:56.915184 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:56.915310 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.915972 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.917327 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.917924 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.917995 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:56.918004 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:56.918165 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 15:06:56.918199 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 15:06:56.918287 30202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:56.914838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:42065 (local address 127.0.236.67:34567)
0806 15:06:56.914885 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:56.914896 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:56.914945 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:56.915099 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:56.915100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:56.915101 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:56.915163 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:56.915164 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.916032 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.916033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:56.917252 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:56.917254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.917993 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.917994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:56.918099 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:56.918101 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 15:06:56.918155 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 15:06:56.918192 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":32}
I20260806 15:06:56.918296 30234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:56.914838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:46541 (local address 127.0.236.67:57971)
0806 15:06:56.915034 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:56.915041 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:56.915088 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:56.915193 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:56.915194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:56.915196 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:56.915250 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:56.915251 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.916032 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.916033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:56.917252 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:56.917254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.917993 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.917994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:56.918097 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:56.918101 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:06:56.918155 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 15:06:56.918191 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":62,"thread_start_us":113,"threads_started":1}
I20260806 15:06:56.918648 29930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa23e369e3f24e0fadb20562a734f16d" candidate_uuid: "a458c962bc604aaeb2afeafefd781cdb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4a91efb8a1c4eac9d8861392e8bdd67" is_pre_election: true
I20260806 15:06:56.918649 29727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa23e369e3f24e0fadb20562a734f16d" candidate_uuid: "a458c962bc604aaeb2afeafefd781cdb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0da1bff1f7b46cdb70ce83d013b3770" is_pre_election: true
I20260806 15:06:56.918762 29930 raft_consensus.cc:2454] T aa23e369e3f24e0fadb20562a734f16d P d4a91efb8a1c4eac9d8861392e8bdd67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a458c962bc604aaeb2afeafefd781cdb in term 0.
I20260806 15:06:56.918772 29727 raft_consensus.cc:2454] T aa23e369e3f24e0fadb20562a734f16d P f0da1bff1f7b46cdb70ce83d013b3770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a458c962bc604aaeb2afeafefd781cdb in term 0.
I20260806 15:06:56.918972 30033 leader_election.cc:302] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a458c962bc604aaeb2afeafefd781cdb, f0da1bff1f7b46cdb70ce83d013b3770; no voters:
I20260806 15:06:56.919099 30230 raft_consensus.cc:2781] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:06:56.919152 30230 raft_consensus.cc:482] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:06:56.919166 30230 raft_consensus.cc:3037] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:56.919771 30230 raft_consensus.cc:504] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4a91efb8a1c4eac9d8861392e8bdd67" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42065 } } peers { permanent_uuid: "f0da1bff1f7b46cdb70ce83d013b3770" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46541 } } peers { permanent_uuid: "a458c962bc604aaeb2afeafefd781cdb" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 36115 } }
I20260806 15:06:56.919881 30230 leader_election.cc:288] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb [CANDIDATE]: Term 1 election: Requested vote from peers d4a91efb8a1c4eac9d8861392e8bdd67 (127.0.236.66:42065), f0da1bff1f7b46cdb70ce83d013b3770 (127.0.236.65:46541)
I20260806 15:06:56.920027 29930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa23e369e3f24e0fadb20562a734f16d" candidate_uuid: "a458c962bc604aaeb2afeafefd781cdb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4a91efb8a1c4eac9d8861392e8bdd67"
I20260806 15:06:56.920048 29727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa23e369e3f24e0fadb20562a734f16d" candidate_uuid: "a458c962bc604aaeb2afeafefd781cdb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0da1bff1f7b46cdb70ce83d013b3770"
I20260806 15:06:56.920099 29930 raft_consensus.cc:3037] T aa23e369e3f24e0fadb20562a734f16d P d4a91efb8a1c4eac9d8861392e8bdd67 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:56.920109 29727 raft_consensus.cc:3037] T aa23e369e3f24e0fadb20562a734f16d P f0da1bff1f7b46cdb70ce83d013b3770 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:56.920928 29930 raft_consensus.cc:2454] T aa23e369e3f24e0fadb20562a734f16d P d4a91efb8a1c4eac9d8861392e8bdd67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a458c962bc604aaeb2afeafefd781cdb in term 1.
I20260806 15:06:56.920928 29727 raft_consensus.cc:2454] T aa23e369e3f24e0fadb20562a734f16d P f0da1bff1f7b46cdb70ce83d013b3770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a458c962bc604aaeb2afeafefd781cdb in term 1.
I20260806 15:06:56.921079 30033 leader_election.cc:302] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a458c962bc604aaeb2afeafefd781cdb, f0da1bff1f7b46cdb70ce83d013b3770; no voters:
I20260806 15:06:56.921156 30230 raft_consensus.cc:2781] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:06:56.921278 30230 raft_consensus.cc:686] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb [term 1 LEADER]: Becoming Leader. State: Replica: a458c962bc604aaeb2afeafefd781cdb, State: Running, Role: LEADER
I20260806 15:06:56.921350 30230 consensus_queue.cc:237] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4a91efb8a1c4eac9d8861392e8bdd67" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42065 } } peers { permanent_uuid: "f0da1bff1f7b46cdb70ce83d013b3770" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46541 } } peers { permanent_uuid: "a458c962bc604aaeb2afeafefd781cdb" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 36115 } }
I20260806 15:06:56.922076 29491 catalog_manager.cc:5495] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb reported cstate change: term changed from 0 to 1, leader changed from <none> to a458c962bc604aaeb2afeafefd781cdb (127.0.236.67). New cstate: current_term: 1 leader_uuid: "a458c962bc604aaeb2afeafefd781cdb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4a91efb8a1c4eac9d8861392e8bdd67" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0da1bff1f7b46cdb70ce83d013b3770" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a458c962bc604aaeb2afeafefd781cdb" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 36115 } health_report { overall_health: HEALTHY } } }
I20260806 15:06:56.942168 30223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:56.940226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42356 (local address 127.0.236.67:36115)
0806 15:06:56.940268 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:56.940270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:56.940277 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:06:56.940316 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:56.940317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:56.940343 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:56.940387 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:06:56.940489 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.940921 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.941568 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:56.941733 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:56.941772 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:56.941866 (+ 94us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:06:56.942002 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:06:56.942060 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:56.942074 (+ 14us) server_negotiation.cc:299] Negotiation successful
0806 15:06:56.942089 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 15:06:56.943614 29727 raft_consensus.cc:1261] T aa23e369e3f24e0fadb20562a734f16d P f0da1bff1f7b46cdb70ce83d013b3770 [term 1 FOLLOWER]: Refusing update from remote peer a458c962bc604aaeb2afeafefd781cdb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:06:56.943614 29930 raft_consensus.cc:1261] T aa23e369e3f24e0fadb20562a734f16d P d4a91efb8a1c4eac9d8861392e8bdd67 [term 1 FOLLOWER]: Refusing update from remote peer a458c962bc604aaeb2afeafefd781cdb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:06:56.943960 30235 consensus_queue.cc:1035] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0da1bff1f7b46cdb70ce83d013b3770" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 46541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:06:56.944078 30230 consensus_queue.cc:1035] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4a91efb8a1c4eac9d8861392e8bdd67" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 42065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:06:56.946252 30239 mvcc.cc:205] Tried to move back new op lower bound from 7315574034198994944 to 7315574034110533632. Current Snapshot: MvccSnapshot[applied={T|T < 7315574034198994944}]
I20260806 15:06:56.948416 29491 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:06:56.948505 29491 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:06:56.949148 29491 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fdb79cddb864445993dc86b61198dcb4: Sending DeleteTablet for 3 replicas of tablet aa23e369e3f24e0fadb20562a734f16d
I20260806 15:06:56.949431 29903 tablet_service.cc:1514] Processing DeleteTablet for tablet aa23e369e3f24e0fadb20562a734f16d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:06:56 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:55346
I20260806 15:06:56.949448 29694 tablet_service.cc:1514] Processing DeleteTablet for tablet aa23e369e3f24e0fadb20562a734f16d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:06:56 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:37402
I20260806 15:06:56.949661 30256 tablet_replica.cc:330] T aa23e369e3f24e0fadb20562a734f16d P f0da1bff1f7b46cdb70ce83d013b3770: stopping tablet replica
I20260806 15:06:56.949697 30103 tablet_service.cc:1514] Processing DeleteTablet for tablet aa23e369e3f24e0fadb20562a734f16d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:06:56 UTC) from {username='kudu', principal='kudu/127.0.236.126@KRBTEST.COM'} at 127.0.0.1:42348
I20260806 15:06:56.949759 30256 raft_consensus.cc:2229] T aa23e369e3f24e0fadb20562a734f16d P f0da1bff1f7b46cdb70ce83d013b3770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:06:56.949846 30256 raft_consensus.cc:2258] T aa23e369e3f24e0fadb20562a734f16d P f0da1bff1f7b46cdb70ce83d013b3770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:06:56.949985 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 29597
I20260806 15:06:56.950481 30255 tablet_replica.cc:330] T aa23e369e3f24e0fadb20562a734f16d P d4a91efb8a1c4eac9d8861392e8bdd67: stopping tablet replica
I20260806 15:06:56.950579 30255 raft_consensus.cc:2229] T aa23e369e3f24e0fadb20562a734f16d P d4a91efb8a1c4eac9d8861392e8bdd67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:06:56.950666 30255 raft_consensus.cc:2258] T aa23e369e3f24e0fadb20562a734f16d P d4a91efb8a1c4eac9d8861392e8bdd67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:06:56.954236 30255 ts_tablet_manager.cc:1905] T aa23e369e3f24e0fadb20562a734f16d P d4a91efb8a1c4eac9d8861392e8bdd67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:06:56.955760 30255 ts_tablet_manager.cc:1918] T aa23e369e3f24e0fadb20562a734f16d P d4a91efb8a1c4eac9d8861392e8bdd67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:06:56.955823 30255 log.cc:1196] T aa23e369e3f24e0fadb20562a734f16d P d4a91efb8a1c4eac9d8861392e8bdd67: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-1/wal/wals/aa23e369e3f24e0fadb20562a734f16d
I20260806 15:06:56.956097 30255 ts_tablet_manager.cc:1939] T aa23e369e3f24e0fadb20562a734f16d P d4a91efb8a1c4eac9d8861392e8bdd67: Deleting consensus metadata
I20260806 15:06:56.960800 29470 catalog_manager.cc:4841] TS d4a91efb8a1c4eac9d8861392e8bdd67 (127.0.236.66:42065): tablet aa23e369e3f24e0fadb20562a734f16d (table test-table [id=03f2bb914c824d6da0f97eec55d4086d]) successfully deleted
I20260806 15:06:56.961845 30257 tablet_replica.cc:330] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb: stopping tablet replica
I20260806 15:06:56.961961 30257 raft_consensus.cc:2229] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:06:56.962052 30257 raft_consensus.cc:2258] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:06:56.962457 30257 ts_tablet_manager.cc:1905] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:06:56.963907 30257 ts_tablet_manager.cc:1918] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:06:56.963979 30257 log.cc:1196] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/aa23e369e3f24e0fadb20562a734f16d
I20260806 15:06:56.964215 30257 ts_tablet_manager.cc:1939] T aa23e369e3f24e0fadb20562a734f16d P a458c962bc604aaeb2afeafefd781cdb: Deleting consensus metadata
I20260806 15:06:56.964596 29470 catalog_manager.cc:4841] TS a458c962bc604aaeb2afeafefd781cdb (127.0.236.67:36115): tablet aa23e369e3f24e0fadb20562a734f16d (table test-table [id=03f2bb914c824d6da0f97eec55d4086d]) successfully deleted
W20260806 15:06:56.966342 29472 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.236.65:46541 (error 108)
I20260806 15:06:56.966815 30218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:56.966697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:46541 (local address 127.0.0.1:37410)
0806 15:06:56.966749 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:56.966784 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.65:46541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 15:06:56.966878 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 29815
W20260806 15:06:56.968304 29472 catalog_manager.cc:4584] TS f0da1bff1f7b46cdb70ce83d013b3770 (127.0.236.65:46541): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet aa23e369e3f24e0fadb20562a734f16d: Network error: Client connection negotiation failed: client connection to 127.0.236.65:46541: connect: Connection refused (error 111)
I20260806 15:06:56.980615 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 30011
I20260806 15:06:56.990026 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 29443
2026-08-06T15:06:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2047 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T15:06:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:06:57Z Disabled control of system clock
I20260806 15:06:57.016705 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:35337
--webserver_interface=127.0.236.126
--webserver_port=0
--builtin_ntp_servers=127.0.236.84:34625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:35337
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 15:06:57.094712 30264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:57.094914 30264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:57.094942 30264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:57.096035 30264 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:06:57.096073 30264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:57.096091 30264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:57.096105 30264 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:06:57.096120 30264 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:06:57.097421 30264 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:34625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:35337
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:35337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:57.097651 30264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:57.097914 30264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:57.102237 30269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:57.102319 30264 server_base.cc:1034] running on GCE node
W20260806 15:06:57.102242 30272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:57.102253 30270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:57.102659 30264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:57.102910 30264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:57.104048 30264 hybrid_clock.cc:648] HybridClock initialized: now 1786028817104033 us; error 27 us; skew 500 ppm
I20260806 15:06:57.105152 30264 webserver.cc:460] Webserver started at http://127.0.236.126:36775/ using document root <none> and password file <none>
I20260806 15:06:57.105350 30264 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:57.105397 30264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:57.105532 30264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:06:57.106523 30264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "cc57cfd236634ddd9d72c1744f27c9e3"
format_stamp: "Formatted at 2026-08-06 15:06:57 on dist-test-slave-tc2s"
server_key: "9af486ddd95a91c301970ea99e81b42b"
server_key_iv: "2df47ac48d9be188dde941aff598f813"
server_key_version: "encryptionkey@0"
I20260806 15:06:57.106830 30264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "cc57cfd236634ddd9d72c1744f27c9e3"
format_stamp: "Formatted at 2026-08-06 15:06:57 on dist-test-slave-tc2s"
server_key: "9af486ddd95a91c301970ea99e81b42b"
server_key_iv: "2df47ac48d9be188dde941aff598f813"
server_key_version: "encryptionkey@0"
I20260806 15:06:57.107950 30264 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:06:57.108572 30279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:57.108714 30264 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:06:57.108794 30264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "cc57cfd236634ddd9d72c1744f27c9e3"
format_stamp: "Formatted at 2026-08-06 15:06:57 on dist-test-slave-tc2s"
server_key: "9af486ddd95a91c301970ea99e81b42b"
server_key_iv: "2df47ac48d9be188dde941aff598f813"
server_key_version: "encryptionkey@0"
I20260806 15:06:57.108850 30264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:57.123438 30264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:57.123675 30264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:57.123788 30264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:57.127285 30264 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:35337
I20260806 15:06:57.127319 30354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:35337 every 8 connection(s)
I20260806 15:06:57.127586 30264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:06:57.128468 30355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:57.130396 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 30264
I20260806 15:06:57.130499 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:06:57.130553 30355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc57cfd236634ddd9d72c1744f27c9e3: Bootstrap starting.
I20260806 15:06:57.130689 945 external_mini_cluster.cc:1428] Setting key b0deacf7f370bbe92bbd2483b4ab9e01
I20260806 15:06:57.131215 30355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc57cfd236634ddd9d72c1744f27c9e3: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:57.131531 30355 log.cc:824] T 00000000000000000000000000000000 P cc57cfd236634ddd9d72c1744f27c9e3: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:57.132256 30355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc57cfd236634ddd9d72c1744f27c9e3: No bootstrap required, opened a new log
I20260806 15:06:57.134435 30355 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc57cfd236634ddd9d72c1744f27c9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc57cfd236634ddd9d72c1744f27c9e3" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35337 } }
I20260806 15:06:57.134588 30355 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc57cfd236634ddd9d72c1744f27c9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:57.134631 30355 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc57cfd236634ddd9d72c1744f27c9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc57cfd236634ddd9d72c1744f27c9e3, State: Initialized, Role: FOLLOWER
I20260806 15:06:57.134745 30355 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc57cfd236634ddd9d72c1744f27c9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc57cfd236634ddd9d72c1744f27c9e3" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35337 } }
I20260806 15:06:57.134812 30355 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc57cfd236634ddd9d72c1744f27c9e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:06:57.134847 30355 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc57cfd236634ddd9d72c1744f27c9e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:06:57.134889 30355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc57cfd236634ddd9d72c1744f27c9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:57.135465 30355 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc57cfd236634ddd9d72c1744f27c9e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc57cfd236634ddd9d72c1744f27c9e3" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35337 } }
I20260806 15:06:57.135564 30355 leader_election.cc:302] T 00000000000000000000000000000000 P cc57cfd236634ddd9d72c1744f27c9e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc57cfd236634ddd9d72c1744f27c9e3; no voters:
I20260806 15:06:57.135730 30355 leader_election.cc:288] T 00000000000000000000000000000000 P cc57cfd236634ddd9d72c1744f27c9e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:06:57.135774 30361 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc57cfd236634ddd9d72c1744f27c9e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:06:57.135900 30361 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc57cfd236634ddd9d72c1744f27c9e3 [term 1 LEADER]: Becoming Leader. State: Replica: cc57cfd236634ddd9d72c1744f27c9e3, State: Running, Role: LEADER
I20260806 15:06:57.135993 30355 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc57cfd236634ddd9d72c1744f27c9e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:06:57.135999 30361 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc57cfd236634ddd9d72c1744f27c9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc57cfd236634ddd9d72c1744f27c9e3" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35337 } }
I20260806 15:06:57.136472 30359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:57.131113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49598 (local address 127.0.236.126:35337)
0806 15:06:57.131307 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:57.131316 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:57.131353 (+ 37us) server_negotiation.cc:407] Connection header received
0806 15:06:57.131850 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:57.131858 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:57.131913 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:57.132047 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:57.132231 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.133610 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.134359 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.134951 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.135080 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:57.135590 (+ 510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:57.135594 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:57.135595 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:06:57.135787 (+ 192us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:57.135830 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:06:57.135845 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:57.135912 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:57.136196 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 15:06:57.136281 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":72,"threads_started":1}
I20260806 15:06:57.136894 30364 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc57cfd236634ddd9d72c1744f27c9e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc57cfd236634ddd9d72c1744f27c9e3. Latest consensus state: current_term: 1 leader_uuid: "cc57cfd236634ddd9d72c1744f27c9e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc57cfd236634ddd9d72c1744f27c9e3" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35337 } } }
I20260806 15:06:57.136976 30364 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc57cfd236634ddd9d72c1744f27c9e3 [sys.catalog]: This master's current role is: LEADER
I20260806 15:06:57.137142 30362 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc57cfd236634ddd9d72c1744f27c9e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc57cfd236634ddd9d72c1744f27c9e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc57cfd236634ddd9d72c1744f27c9e3" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35337 } } }
I20260806 15:06:57.137275 30362 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc57cfd236634ddd9d72c1744f27c9e3 [sys.catalog]: This master's current role is: LEADER
W20260806 15:06:57.137717 30381 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cc57cfd236634ddd9d72c1744f27c9e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:06:57.137759 30381 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:06:57.137825 30375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:06:57.138036 30375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:06:57.139439 30375 catalog_manager.cc:1347] Generated new cluster ID: 3d4f71ffea034fb492d7458492985290
I20260806 15:06:57.139492 30375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:06:57.153424 30375 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:06:57.153875 30375 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:06:57.160243 30375 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc57cfd236634ddd9d72c1744f27c9e3: Generated new TSK 0
I20260806 15:06:57.160394 30375 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:06:57.167201 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.65:0
--local_ip_for_outbound_sockets=127.0.236.65
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:35337
--builtin_ntp_servers=127.0.236.84:34625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 15:06:57.246315 30388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:57.246508 30388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:57.246528 30388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:57.247629 30388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:57.247666 30388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:57.247689 30388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.65
I20260806 15:06:57.248994 30388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:34625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.236.65
--webserver_port=0
--tserver_master_addrs=127.0.236.126:35337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.65
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:57.249202 30388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:57.249481 30388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:57.254925 30393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:57.254922 30399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:57.254985 30397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:57.255245 30388 server_base.cc:1034] running on GCE node
I20260806 15:06:57.255435 30388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:57.255710 30388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:57.256867 30388 hybrid_clock.cc:648] HybridClock initialized: now 1786028817256850 us; error 33 us; skew 500 ppm
I20260806 15:06:57.257969 30388 webserver.cc:460] Webserver started at http://127.0.236.65:44321/ using document root <none> and password file <none>
I20260806 15:06:57.258170 30388 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:57.258210 30388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:57.258302 30388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:06:57.259243 30388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-0/data/instance:
uuid: "d21e5a01a6bb4ceab3fc3009e5912498"
format_stamp: "Formatted at 2026-08-06 15:06:57 on dist-test-slave-tc2s"
server_key: "e028c8cd8fd873c7577893326355e6a2"
server_key_iv: "7c263121641bf8b938a4dea2048f27ce"
server_key_version: "encryptionkey@0"
I20260806 15:06:57.259526 30388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-0/wal/instance:
uuid: "d21e5a01a6bb4ceab3fc3009e5912498"
format_stamp: "Formatted at 2026-08-06 15:06:57 on dist-test-slave-tc2s"
server_key: "e028c8cd8fd873c7577893326355e6a2"
server_key_iv: "7c263121641bf8b938a4dea2048f27ce"
server_key_version: "encryptionkey@0"
I20260806 15:06:57.260640 30388 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 15:06:57.261251 30408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:57.261406 30388 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:06:57.261488 30388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-0/wal
uuid: "d21e5a01a6bb4ceab3fc3009e5912498"
format_stamp: "Formatted at 2026-08-06 15:06:57 on dist-test-slave-tc2s"
server_key: "e028c8cd8fd873c7577893326355e6a2"
server_key_iv: "7c263121641bf8b938a4dea2048f27ce"
server_key_version: "encryptionkey@0"
I20260806 15:06:57.261562 30388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:57.279640 30388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:57.279877 30388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:57.279995 30388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:57.280182 30388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:06:57.280480 30388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:06:57.280521 30388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:57.280548 30388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:06:57.280570 30388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:57.285921 30388 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.65:35505
I20260806 15:06:57.285977 30591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.65:35505 every 8 connection(s)
I20260806 15:06:57.286253 30388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-0/data/info.pb
I20260806 15:06:57.290606 30359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:57.286889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.65:45051 (local address 127.0.236.126:35337)
0806 15:06:57.286951 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:57.286953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:57.287153 (+ 200us) server_negotiation.cc:407] Connection header received
0806 15:06:57.287583 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:57.287584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:57.287593 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:57.287641 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:57.288216 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.288692 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.289550 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.289721 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.289759 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:57.290407 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:57.290409 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:57.290409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:06:57.290414 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:57.290429 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:06:57.290433 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:57.290471 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:57.290531 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 15:06:57.290551 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 15:06:57.290799 30595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:57.286871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35337 (local address 127.0.236.65:45051)
0806 15:06:57.287045 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:57.287082 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:57.287505 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:57.287690 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:57.287693 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:57.287701 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:06:57.288160 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:57.288164 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.288750 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.288750 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:57.289481 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:57.289483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.289769 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.289770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:57.289868 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:57.290280 (+ 412us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:06:57.290280 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:06:57.290313 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:06:57.290335 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:06:57.290473 (+ 138us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:06:57.290475 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:06:57.290477 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 15:06:57.290529 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 15:06:57.290601 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
I20260806 15:06:57.291204 30592 heartbeater.cc:347] Connected to a master server at 127.0.236.126:35337
I20260806 15:06:57.291287 30592 heartbeater.cc:464] Registering TS with master...
I20260806 15:06:57.291445 30592 heartbeater.cc:511] Master 127.0.236.126:35337 requested a full tablet report, sending...
I20260806 15:06:57.291889 30305 ts_manager.cc:194] Registered new tserver with Master: d21e5a01a6bb4ceab3fc3009e5912498 (127.0.236.65:35505)
I20260806 15:06:57.292294 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 30388
I20260806 15:06:57.292392 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-0/wal/instance
I20260806 15:06:57.292569 945 external_mini_cluster.cc:1428] Setting key ca02e2e7a5f259ed7d52b918497fcc88
I20260806 15:06:57.292687 30305 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.236.65:45051
I20260806 15:06:57.293464 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.66:0
--local_ip_for_outbound_sockets=127.0.236.66
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:35337
--builtin_ntp_servers=127.0.236.84:34625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 15:06:57.372680 30597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:57.372861 30597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:57.372880 30597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:57.373941 30597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:57.373971 30597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:57.373993 30597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.66
I20260806 15:06:57.375351 30597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:34625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.236.66
--webserver_port=0
--tserver_master_addrs=127.0.236.126:35337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.66
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:57.375524 30597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:57.375737 30597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:57.380239 30611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:57.380214 30609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:57.380429 30597 server_base.cc:1034] running on GCE node
W20260806 15:06:57.380537 30604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:57.380731 30597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:57.380950 30597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:57.382079 30597 hybrid_clock.cc:648] HybridClock initialized: now 1786028817382067 us; error 30 us; skew 500 ppm
I20260806 15:06:57.383114 30597 webserver.cc:460] Webserver started at http://127.0.236.66:40473/ using document root <none> and password file <none>
I20260806 15:06:57.383309 30597 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:57.383349 30597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:57.383478 30597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:06:57.384416 30597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-1/data/instance:
uuid: "768d9ca1b6454fe199d1a35b48c38fba"
format_stamp: "Formatted at 2026-08-06 15:06:57 on dist-test-slave-tc2s"
server_key: "73ee672f04cfccbfedcbcd1651d82333"
server_key_iv: "535e4df2067d6770ee68878c20ebf0de"
server_key_version: "encryptionkey@0"
I20260806 15:06:57.384718 30597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-1/wal/instance:
uuid: "768d9ca1b6454fe199d1a35b48c38fba"
format_stamp: "Formatted at 2026-08-06 15:06:57 on dist-test-slave-tc2s"
server_key: "73ee672f04cfccbfedcbcd1651d82333"
server_key_iv: "535e4df2067d6770ee68878c20ebf0de"
server_key_version: "encryptionkey@0"
I20260806 15:06:57.385869 30597 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 15:06:57.386528 30617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:57.386658 30597 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:06:57.386737 30597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-1/wal
uuid: "768d9ca1b6454fe199d1a35b48c38fba"
format_stamp: "Formatted at 2026-08-06 15:06:57 on dist-test-slave-tc2s"
server_key: "73ee672f04cfccbfedcbcd1651d82333"
server_key_iv: "535e4df2067d6770ee68878c20ebf0de"
server_key_version: "encryptionkey@0"
I20260806 15:06:57.386802 30597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:57.415558 30597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:57.415800 30597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:57.415917 30597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:57.416111 30597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:06:57.416401 30597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:06:57.416435 30597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:57.416462 30597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:06:57.416486 30597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:57.421772 30597 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.66:37591
I20260806 15:06:57.421811 30778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.66:37591 every 8 connection(s)
I20260806 15:06:57.422102 30597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-1/data/info.pb
I20260806 15:06:57.426568 30359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:57.422724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.66:58623 (local address 127.0.236.126:35337)
0806 15:06:57.422765 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:57.422767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:57.422974 (+ 207us) server_negotiation.cc:407] Connection header received
0806 15:06:57.423412 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:57.423413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:57.423422 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:57.423474 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:57.424103 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.424597 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.425476 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.425651 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.425695 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:57.426360 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:57.426361 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:57.426361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:06:57.426365 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:57.426392 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:06:57.426396 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:57.426434 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:57.426490 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 15:06:57.426510 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 15:06:57.426728 30785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:57.422696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35337 (local address 127.0.236.66:58623)
0806 15:06:57.422870 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:57.422908 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:57.423335 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:57.423521 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:57.423524 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:57.423532 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:06:57.424031 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:57.424036 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.424658 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.424659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:57.425418 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:57.425421 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.425702 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.425703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:57.425798 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:57.426245 (+ 447us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:06:57.426245 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:06:57.426271 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:06:57.426292 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:06:57.426437 (+ 145us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:06:57.426439 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:06:57.426441 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 15:06:57.426483 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 15:06:57.426554 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":68,"threads_started":1}
I20260806 15:06:57.427150 30779 heartbeater.cc:347] Connected to a master server at 127.0.236.126:35337
I20260806 15:06:57.427232 30779 heartbeater.cc:464] Registering TS with master...
I20260806 15:06:57.427392 30779 heartbeater.cc:511] Master 127.0.236.126:35337 requested a full tablet report, sending...
I20260806 15:06:57.427706 30304 ts_manager.cc:194] Registered new tserver with Master: 768d9ca1b6454fe199d1a35b48c38fba (127.0.236.66:37591)
I20260806 15:06:57.428061 30304 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.236.66:58623
I20260806 15:06:57.428407 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 30597
I20260806 15:06:57.428517 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-1/wal/instance
I20260806 15:06:57.428695 945 external_mini_cluster.cc:1428] Setting key 59c44d052ee5e695c7e1e73c7bf20919
I20260806 15:06:57.429466 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.236.67:0
--local_ip_for_outbound_sockets=127.0.236.67
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:35337
--builtin_ntp_servers=127.0.236.84:34625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 15:06:57.509039 30787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:57.509215 30787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:57.509243 30787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:57.510382 30787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:57.510418 30787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:57.510452 30787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.236.67
I20260806 15:06:57.511785 30787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:34625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.236.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.236.67
--webserver_port=0
--tserver_master_addrs=127.0.236.126:35337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.236.67
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:57.511996 30787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:57.512254 30787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:57.516759 30793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:57.516783 30795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:57.516935 30787 server_base.cc:1034] running on GCE node
W20260806 15:06:57.517019 30798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:57.517216 30787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:57.517443 30787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:57.518574 30787 hybrid_clock.cc:648] HybridClock initialized: now 1786028817518562 us; error 29 us; skew 500 ppm
I20260806 15:06:57.519687 30787 webserver.cc:460] Webserver started at http://127.0.236.67:46809/ using document root <none> and password file <none>
I20260806 15:06:57.519910 30787 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:57.519953 30787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:57.520073 30787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:06:57.521045 30787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-2/data/instance:
uuid: "8baac205da574e5d9b44942f8a8a04ec"
format_stamp: "Formatted at 2026-08-06 15:06:57 on dist-test-slave-tc2s"
server_key: "867b7a6580a51ec5024438e90ae3d7ec"
server_key_iv: "95c8361728598e8c9df418f5e8a095f0"
server_key_version: "encryptionkey@0"
I20260806 15:06:57.521344 30787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-2/wal/instance:
uuid: "8baac205da574e5d9b44942f8a8a04ec"
format_stamp: "Formatted at 2026-08-06 15:06:57 on dist-test-slave-tc2s"
server_key: "867b7a6580a51ec5024438e90ae3d7ec"
server_key_iv: "95c8361728598e8c9df418f5e8a095f0"
server_key_version: "encryptionkey@0"
I20260806 15:06:57.522544 30787 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:06:57.523151 30806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:57.523284 30787 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:06:57.523355 30787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-2/wal
uuid: "8baac205da574e5d9b44942f8a8a04ec"
format_stamp: "Formatted at 2026-08-06 15:06:57 on dist-test-slave-tc2s"
server_key: "867b7a6580a51ec5024438e90ae3d7ec"
server_key_iv: "95c8361728598e8c9df418f5e8a095f0"
server_key_version: "encryptionkey@0"
I20260806 15:06:57.523412 30787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:57.537494 30787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:57.537712 30787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:57.537820 30787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:57.538024 30787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:06:57.538287 30787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:06:57.538319 30787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:57.538345 30787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:06:57.538367 30787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:57.543849 30787 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.67:38457
I20260806 15:06:57.543885 30974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.67:38457 every 8 connection(s)
I20260806 15:06:57.544178 30787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-2/data/info.pb
I20260806 15:06:57.548806 30359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:57.544798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:36215 (local address 127.0.236.126:35337)
0806 15:06:57.544837 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:57.544839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:57.545047 (+ 208us) server_negotiation.cc:407] Connection header received
0806 15:06:57.545495 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:57.545496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:57.545510 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:57.545559 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:57.546226 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.546712 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.547632 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.547822 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.547876 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:57.548559 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:57.548561 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:57.548561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:06:57.548566 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:57.548582 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:06:57.548586 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:57.548627 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:57.548699 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 15:06:57.548727 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 15:06:57.548980 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:57.544772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:35337 (local address 127.0.236.67:36215)
0806 15:06:57.544946 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:57.544975 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:57.545401 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:57.545621 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:57.545625 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:57.545634 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:06:57.546143 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:57.546148 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.546789 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.546790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:57.547554 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:57.547556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.547887 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.547888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:57.548014 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:57.548438 (+ 424us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:06:57.548438 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:06:57.548463 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:06:57.548484 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:06:57.548633 (+ 149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:06:57.548637 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:06:57.548640 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:06:57.548697 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 15:06:57.548790 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260806 15:06:57.549401 30976 heartbeater.cc:347] Connected to a master server at 127.0.236.126:35337
I20260806 15:06:57.549484 30976 heartbeater.cc:464] Registering TS with master...
I20260806 15:06:57.549641 30976 heartbeater.cc:511] Master 127.0.236.126:35337 requested a full tablet report, sending...
I20260806 15:06:57.549974 30304 ts_manager.cc:194] Registered new tserver with Master: 8baac205da574e5d9b44942f8a8a04ec (127.0.236.67:38457)
I20260806 15:06:57.550334 30304 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.236.67:36215
I20260806 15:06:57.554266 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 30787
I20260806 15:06:57.554345 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-2/wal/instance
I20260806 15:06:57.554520 945 external_mini_cluster.cc:1428] Setting key ac51504faa8f34ef286e12c320c9fdc6
I20260806 15:06:57.555318 945 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:06:57.559033 30359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:57.556565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49600 (local address 127.0.236.126:35337)
0806 15:06:57.556605 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:57.556607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:57.556709 (+ 102us) server_negotiation.cc:407] Connection header received
0806 15:06:57.556749 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:57.556750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:57.556757 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:57.556799 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:57.556925 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.557363 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.558094 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.558268 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.558313 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:57.558868 (+ 555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:57.558869 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:57.558869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:06:57.558872 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:57.558884 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:06:57.558887 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:57.558929 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:57.558967 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 15:06:57.558984 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 15:06:57.560314 30304 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49600:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:06:57.561784 30304 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:06:57.567706 30989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:57.563662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:37591 (local address 127.0.0.1:47570)
0806 15:06:57.564403 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:57.564412 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:57.564460 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:57.564710 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:57.564711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:57.564714 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:57.564747 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:57.564748 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.565927 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.565928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:57.566807 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:57.566809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.567429 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.567430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:57.567506 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:57.567519 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 15:06:57.567586 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 15:06:57.567635 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"mutex_wait_us":5,"thread_start_us":101,"threads_started":1}
I20260806 15:06:57.567716 30992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:57.563895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47570 (local address 127.0.236.66:37591)
0806 15:06:57.564088 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:57.564092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:57.564515 (+ 423us) server_negotiation.cc:407] Connection header received
0806 15:06:57.564561 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:57.564563 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:57.564577 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:57.564643 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:57.564821 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.565840 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.566877 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.567349 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.567411 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:57.567505 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:57.567600 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 15:06:57.567636 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":106,"threads_started":1}
I20260806 15:06:57.567889 30988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:57.563649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:35505 (local address 127.0.0.1:46384)
0806 15:06:57.563799 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:57.563833 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:57.563895 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:57.564325 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:57.564332 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:57.564334 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:57.564445 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:57.564447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.565486 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.565487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:57.566808 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:57.566810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.567690 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.567691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:57.567751 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:57.567755 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:06:57.567783 (+ 28us) client_negotiation.cc:240] Negotiation successful
0806 15:06:57.567820 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":22,"thread_start_us":57,"threads_started":1}
I20260806 15:06:57.568001 30993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:57.564017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55746 (local address 127.0.236.67:38457)
0806 15:06:57.564176 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:57.564180 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:57.564189 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:06:57.564221 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:57.564223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:57.564236 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:57.564301 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:57.564508 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.565553 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.566902 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.567592 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.567658 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:57.567733 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:57.567887 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 15:06:57.567930 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":78,"threads_started":1}
I20260806 15:06:57.568058 30990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:57.563650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.67:38457 (local address 127.0.0.1:55746)
0806 15:06:57.563906 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:57.563916 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:57.563969 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:57.564324 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:57.564327 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:57.564329 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:57.564387 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:57.564389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.565639 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.565640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:57.566808 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:57.566811 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.567762 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.567763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:57.567813 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:57.567816 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:06:57.567865 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 15:06:57.567895 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":24,"thread_start_us":73,"threads_started":1}
I20260806 15:06:57.568434 30991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:57.563825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46384 (local address 127.0.236.65:35505)
0806 15:06:57.564188 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:57.564191 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:57.564201 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:06:57.564233 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:57.564235 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:57.564246 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:57.564312 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:57.564551 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.565416 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.566903 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.567592 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.568232 (+ 640us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:57.568299 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:57.568322 (+ 23us) server_negotiation.cc:299] Negotiation successful
0806 15:06:57.568359 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":100,"threads_started":1}
I20260806 15:06:57.568738 30878 tablet_service.cc:1467] Processing CreateTablet for tablet 56699fb289a74ae989fdb30ab5071e7b (DEFAULT_TABLE table=test-table [id=a515ad37b09d45a5b03bd6530aaa1db6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:06:57.568740 30684 tablet_service.cc:1467] Processing CreateTablet for tablet 56699fb289a74ae989fdb30ab5071e7b (DEFAULT_TABLE table=test-table [id=a515ad37b09d45a5b03bd6530aaa1db6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:06:57.569073 30684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56699fb289a74ae989fdb30ab5071e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:57.569074 30878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56699fb289a74ae989fdb30ab5071e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:57.569051 30491 tablet_service.cc:1467] Processing CreateTablet for tablet 56699fb289a74ae989fdb30ab5071e7b (DEFAULT_TABLE table=test-table [id=a515ad37b09d45a5b03bd6530aaa1db6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:06:57.569337 30491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56699fb289a74ae989fdb30ab5071e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:57.571663 30996 tablet_bootstrap.cc:492] T 56699fb289a74ae989fdb30ab5071e7b P 768d9ca1b6454fe199d1a35b48c38fba: Bootstrap starting.
I20260806 15:06:57.572157 30997 tablet_bootstrap.cc:492] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec: Bootstrap starting.
I20260806 15:06:57.572232 30996 tablet_bootstrap.cc:654] T 56699fb289a74ae989fdb30ab5071e7b P 768d9ca1b6454fe199d1a35b48c38fba: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:57.572374 30999 tablet_bootstrap.cc:492] T 56699fb289a74ae989fdb30ab5071e7b P d21e5a01a6bb4ceab3fc3009e5912498: Bootstrap starting.
I20260806 15:06:57.572549 30996 log.cc:824] T 56699fb289a74ae989fdb30ab5071e7b P 768d9ca1b6454fe199d1a35b48c38fba: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:57.572747 30997 tablet_bootstrap.cc:654] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:57.572961 30999 tablet_bootstrap.cc:654] T 56699fb289a74ae989fdb30ab5071e7b P d21e5a01a6bb4ceab3fc3009e5912498: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:57.573030 30997 log.cc:824] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:57.573251 30999 log.cc:824] T 56699fb289a74ae989fdb30ab5071e7b P d21e5a01a6bb4ceab3fc3009e5912498: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:57.573333 30996 tablet_bootstrap.cc:492] T 56699fb289a74ae989fdb30ab5071e7b P 768d9ca1b6454fe199d1a35b48c38fba: No bootstrap required, opened a new log
I20260806 15:06:57.573395 30996 ts_tablet_manager.cc:1397] T 56699fb289a74ae989fdb30ab5071e7b P 768d9ca1b6454fe199d1a35b48c38fba: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:06:57.573729 30997 tablet_bootstrap.cc:492] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec: No bootstrap required, opened a new log
I20260806 15:06:57.573799 30997 ts_tablet_manager.cc:1397] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:06:57.573920 30999 tablet_bootstrap.cc:492] T 56699fb289a74ae989fdb30ab5071e7b P d21e5a01a6bb4ceab3fc3009e5912498: No bootstrap required, opened a new log
I20260806 15:06:57.573983 30999 ts_tablet_manager.cc:1397] T 56699fb289a74ae989fdb30ab5071e7b P d21e5a01a6bb4ceab3fc3009e5912498: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:06:57.575672 30996 raft_consensus.cc:348] T 56699fb289a74ae989fdb30ab5071e7b P 768d9ca1b6454fe199d1a35b48c38fba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8baac205da574e5d9b44942f8a8a04ec" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38457 } } peers { permanent_uuid: "d21e5a01a6bb4ceab3fc3009e5912498" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 35505 } } peers { permanent_uuid: "768d9ca1b6454fe199d1a35b48c38fba" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37591 } }
I20260806 15:06:57.575832 30996 raft_consensus.cc:374] T 56699fb289a74ae989fdb30ab5071e7b P 768d9ca1b6454fe199d1a35b48c38fba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:57.575865 30996 raft_consensus.cc:729] T 56699fb289a74ae989fdb30ab5071e7b P 768d9ca1b6454fe199d1a35b48c38fba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 768d9ca1b6454fe199d1a35b48c38fba, State: Initialized, Role: FOLLOWER
I20260806 15:06:57.575986 30996 consensus_queue.cc:260] T 56699fb289a74ae989fdb30ab5071e7b P 768d9ca1b6454fe199d1a35b48c38fba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8baac205da574e5d9b44942f8a8a04ec" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38457 } } peers { permanent_uuid: "d21e5a01a6bb4ceab3fc3009e5912498" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 35505 } } peers { permanent_uuid: "768d9ca1b6454fe199d1a35b48c38fba" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37591 } }
I20260806 15:06:57.576226 30996 ts_tablet_manager.cc:1428] T 56699fb289a74ae989fdb30ab5071e7b P 768d9ca1b6454fe199d1a35b48c38fba: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 15:06:57.576333 30779 heartbeater.cc:503] Master 127.0.236.126:35337 was elected leader, sending a full tablet report...
I20260806 15:06:57.576262 30999 raft_consensus.cc:348] T 56699fb289a74ae989fdb30ab5071e7b P d21e5a01a6bb4ceab3fc3009e5912498 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8baac205da574e5d9b44942f8a8a04ec" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38457 } } peers { permanent_uuid: "d21e5a01a6bb4ceab3fc3009e5912498" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 35505 } } peers { permanent_uuid: "768d9ca1b6454fe199d1a35b48c38fba" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37591 } }
I20260806 15:06:57.576399 30999 raft_consensus.cc:374] T 56699fb289a74ae989fdb30ab5071e7b P d21e5a01a6bb4ceab3fc3009e5912498 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:57.576447 30999 raft_consensus.cc:729] T 56699fb289a74ae989fdb30ab5071e7b P d21e5a01a6bb4ceab3fc3009e5912498 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d21e5a01a6bb4ceab3fc3009e5912498, State: Initialized, Role: FOLLOWER
I20260806 15:06:57.576785 30997 raft_consensus.cc:348] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8baac205da574e5d9b44942f8a8a04ec" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38457 } } peers { permanent_uuid: "d21e5a01a6bb4ceab3fc3009e5912498" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 35505 } } peers { permanent_uuid: "768d9ca1b6454fe199d1a35b48c38fba" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37591 } }
I20260806 15:06:57.576931 30997 raft_consensus.cc:374] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:57.576967 30997 raft_consensus.cc:729] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8baac205da574e5d9b44942f8a8a04ec, State: Initialized, Role: FOLLOWER
I20260806 15:06:57.577093 30997 consensus_queue.cc:260] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8baac205da574e5d9b44942f8a8a04ec" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38457 } } peers { permanent_uuid: "d21e5a01a6bb4ceab3fc3009e5912498" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 35505 } } peers { permanent_uuid: "768d9ca1b6454fe199d1a35b48c38fba" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37591 } }
I20260806 15:06:57.577327 30997 ts_tablet_manager.cc:1428] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 15:06:57.577428 30976 heartbeater.cc:503] Master 127.0.236.126:35337 was elected leader, sending a full tablet report...
I20260806 15:06:57.576548 30999 consensus_queue.cc:260] T 56699fb289a74ae989fdb30ab5071e7b P d21e5a01a6bb4ceab3fc3009e5912498 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8baac205da574e5d9b44942f8a8a04ec" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38457 } } peers { permanent_uuid: "d21e5a01a6bb4ceab3fc3009e5912498" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 35505 } } peers { permanent_uuid: "768d9ca1b6454fe199d1a35b48c38fba" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37591 } }
I20260806 15:06:57.578349 30999 ts_tablet_manager.cc:1428] T 56699fb289a74ae989fdb30ab5071e7b P d21e5a01a6bb4ceab3fc3009e5912498: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 15:06:57.578451 30592 heartbeater.cc:503] Master 127.0.236.126:35337 was elected leader, sending a full tablet report...
W20260806 15:06:57.672575 30782 tablet.cc:2369] T 56699fb289a74ae989fdb30ab5071e7b P 768d9ca1b6454fe199d1a35b48c38fba: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 15:06:57.786866 30593 tablet.cc:2369] T 56699fb289a74ae989fdb30ab5071e7b P d21e5a01a6bb4ceab3fc3009e5912498: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 15:06:57.794633 30977 tablet.cc:2369] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:06:57.822738 31007 raft_consensus.cc:482] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:06:57.822835 31007 raft_consensus.cc:504] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8baac205da574e5d9b44942f8a8a04ec" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38457 } } peers { permanent_uuid: "d21e5a01a6bb4ceab3fc3009e5912498" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 35505 } } peers { permanent_uuid: "768d9ca1b6454fe199d1a35b48c38fba" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37591 } }
I20260806 15:06:57.823189 31007 leader_election.cc:288] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d21e5a01a6bb4ceab3fc3009e5912498 (127.0.236.65:35505), 768d9ca1b6454fe199d1a35b48c38fba (127.0.236.66:37591)
I20260806 15:06:57.826589 30992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:57.823420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:45313 (local address 127.0.236.66:37591)
0806 15:06:57.823472 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:57.823475 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:57.823689 (+ 214us) server_negotiation.cc:407] Connection header received
0806 15:06:57.823739 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:57.823740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:57.823749 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:57.823799 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:57.823910 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.824584 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.825867 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.826258 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.826315 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:57.826323 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:57.826482 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 15:06:57.826523 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 15:06:57.826606 30991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:57.823435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.236.67:60237 (local address 127.0.236.65:35505)
0806 15:06:57.823481 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:57.823483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:57.823494 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:06:57.823524 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:57.823525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:57.823536 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:57.823596 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:06:57.823752 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.824513 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.825936 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.826284 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.826350 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:57.826358 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:57.826505 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 15:06:57.826543 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 15:06:57.826637 31011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:57.823337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:37591 (local address 127.0.236.67:45313)
0806 15:06:57.823618 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:57.823628 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:57.823683 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:57.823808 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:57.823809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:57.823810 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:57.823856 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:57.823857 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.824641 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.824642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:57.825808 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:57.825810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.826343 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.826344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:57.826417 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:57.826421 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:06:57.826462 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 15:06:57.826503 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":144,"thread_start_us":99,"threads_started":1}
I20260806 15:06:57.826638 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:57.823336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:35505 (local address 127.0.236.67:60237)
0806 15:06:57.823389 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:57.823400 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:06:57.823458 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:06:57.823607 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:06:57.823608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:06:57.823610 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:06:57.823683 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:57.823685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.824586 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.824586 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:57.825796 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:06:57.825799 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.826363 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.826364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:06:57.826435 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:57.826438 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:06:57.826489 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 15:06:57.826529 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":2}
I20260806 15:06:57.826975 30717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56699fb289a74ae989fdb30ab5071e7b" candidate_uuid: "8baac205da574e5d9b44942f8a8a04ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "768d9ca1b6454fe199d1a35b48c38fba" is_pre_election: true
I20260806 15:06:57.827126 30717 raft_consensus.cc:2454] T 56699fb289a74ae989fdb30ab5071e7b P 768d9ca1b6454fe199d1a35b48c38fba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8baac205da574e5d9b44942f8a8a04ec in term 0.
I20260806 15:06:57.827098 30523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56699fb289a74ae989fdb30ab5071e7b" candidate_uuid: "8baac205da574e5d9b44942f8a8a04ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d21e5a01a6bb4ceab3fc3009e5912498" is_pre_election: true
I20260806 15:06:57.827234 30523 raft_consensus.cc:2454] T 56699fb289a74ae989fdb30ab5071e7b P d21e5a01a6bb4ceab3fc3009e5912498 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8baac205da574e5d9b44942f8a8a04ec in term 0.
I20260806 15:06:57.827346 30811 leader_election.cc:302] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 768d9ca1b6454fe199d1a35b48c38fba, 8baac205da574e5d9b44942f8a8a04ec; no voters:
I20260806 15:06:57.827477 31007 raft_consensus.cc:2781] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:06:57.827540 31007 raft_consensus.cc:482] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:06:57.827556 31007 raft_consensus.cc:3037] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:57.828203 31007 raft_consensus.cc:504] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8baac205da574e5d9b44942f8a8a04ec" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38457 } } peers { permanent_uuid: "d21e5a01a6bb4ceab3fc3009e5912498" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 35505 } } peers { permanent_uuid: "768d9ca1b6454fe199d1a35b48c38fba" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37591 } }
I20260806 15:06:57.828316 31007 leader_election.cc:288] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec [CANDIDATE]: Term 1 election: Requested vote from peers d21e5a01a6bb4ceab3fc3009e5912498 (127.0.236.65:35505), 768d9ca1b6454fe199d1a35b48c38fba (127.0.236.66:37591)
I20260806 15:06:57.828470 30717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56699fb289a74ae989fdb30ab5071e7b" candidate_uuid: "8baac205da574e5d9b44942f8a8a04ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "768d9ca1b6454fe199d1a35b48c38fba"
I20260806 15:06:57.828540 30717 raft_consensus.cc:3037] T 56699fb289a74ae989fdb30ab5071e7b P 768d9ca1b6454fe199d1a35b48c38fba [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:57.829192 30717 raft_consensus.cc:2454] T 56699fb289a74ae989fdb30ab5071e7b P 768d9ca1b6454fe199d1a35b48c38fba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8baac205da574e5d9b44942f8a8a04ec in term 1.
I20260806 15:06:57.829285 30523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56699fb289a74ae989fdb30ab5071e7b" candidate_uuid: "8baac205da574e5d9b44942f8a8a04ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d21e5a01a6bb4ceab3fc3009e5912498"
I20260806 15:06:57.829341 30811 leader_election.cc:302] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 768d9ca1b6454fe199d1a35b48c38fba, 8baac205da574e5d9b44942f8a8a04ec; no voters:
I20260806 15:06:57.829351 30523 raft_consensus.cc:3037] T 56699fb289a74ae989fdb30ab5071e7b P d21e5a01a6bb4ceab3fc3009e5912498 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:57.829420 31007 raft_consensus.cc:2781] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:06:57.829545 31007 raft_consensus.cc:686] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec [term 1 LEADER]: Becoming Leader. State: Replica: 8baac205da574e5d9b44942f8a8a04ec, State: Running, Role: LEADER
I20260806 15:06:57.829635 31007 consensus_queue.cc:237] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8baac205da574e5d9b44942f8a8a04ec" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38457 } } peers { permanent_uuid: "d21e5a01a6bb4ceab3fc3009e5912498" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 35505 } } peers { permanent_uuid: "768d9ca1b6454fe199d1a35b48c38fba" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37591 } }
I20260806 15:06:57.830183 30523 raft_consensus.cc:2454] T 56699fb289a74ae989fdb30ab5071e7b P d21e5a01a6bb4ceab3fc3009e5912498 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8baac205da574e5d9b44942f8a8a04ec in term 1.
I20260806 15:06:57.830694 30305 catalog_manager.cc:5495] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 8baac205da574e5d9b44942f8a8a04ec (127.0.236.67). New cstate: current_term: 1 leader_uuid: "8baac205da574e5d9b44942f8a8a04ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8baac205da574e5d9b44942f8a8a04ec" member_type: VOTER last_known_addr { host: "127.0.236.67" port: 38457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d21e5a01a6bb4ceab3fc3009e5912498" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 35505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "768d9ca1b6454fe199d1a35b48c38fba" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37591 } health_report { overall_health: UNKNOWN } } }
I20260806 15:06:57.850703 30993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:57.848492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55756 (local address 127.0.236.67:38457)
0806 15:06:57.848548 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:57.848550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:57.848568 (+ 18us) server_negotiation.cc:407] Connection header received
0806 15:06:57.848598 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:57.848598 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:57.848615 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:57.848669 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:06:57.848791 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.849236 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.849985 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:57.850187 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:57.850239 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:57.850393 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:06:57.850487 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:06:57.850556 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:57.850602 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 15:06:57.850619 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 15:06:57.852222 30717 raft_consensus.cc:1261] T 56699fb289a74ae989fdb30ab5071e7b P 768d9ca1b6454fe199d1a35b48c38fba [term 1 FOLLOWER]: Refusing update from remote peer 8baac205da574e5d9b44942f8a8a04ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:06:57.852236 30523 raft_consensus.cc:1261] T 56699fb289a74ae989fdb30ab5071e7b P d21e5a01a6bb4ceab3fc3009e5912498 [term 1 FOLLOWER]: Refusing update from remote peer 8baac205da574e5d9b44942f8a8a04ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:06:57.852586 31012 consensus_queue.cc:1035] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "d21e5a01a6bb4ceab3fc3009e5912498" member_type: VOTER last_known_addr { host: "127.0.236.65" port: 35505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:06:57.852674 31007 consensus_queue.cc:1035] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "768d9ca1b6454fe199d1a35b48c38fba" member_type: VOTER last_known_addr { host: "127.0.236.66" port: 37591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:06:57.857079 30305 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:06:57.857159 30305 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:06:57.857844 30305 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cc57cfd236634ddd9d72c1744f27c9e3: Sending DeleteTablet for 3 replicas of tablet 56699fb289a74ae989fdb30ab5071e7b
I20260806 15:06:57.858170 30878 tablet_service.cc:1514] Processing DeleteTablet for tablet 56699fb289a74ae989fdb30ab5071e7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:06:57 UTC) from {username='slave'} at 127.0.0.1:55746
I20260806 15:06:57.858201 30491 tablet_service.cc:1514] Processing DeleteTablet for tablet 56699fb289a74ae989fdb30ab5071e7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:06:57 UTC) from {username='slave'} at 127.0.0.1:46384
I20260806 15:06:57.858451 31029 tablet_replica.cc:330] T 56699fb289a74ae989fdb30ab5071e7b P d21e5a01a6bb4ceab3fc3009e5912498: stopping tablet replica
I20260806 15:06:57.858450 30684 tablet_service.cc:1514] Processing DeleteTablet for tablet 56699fb289a74ae989fdb30ab5071e7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:06:57 UTC) from {username='slave'} at 127.0.0.1:47570
I20260806 15:06:57.858487 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 30388
I20260806 15:06:57.858553 31029 raft_consensus.cc:2229] T 56699fb289a74ae989fdb30ab5071e7b P d21e5a01a6bb4ceab3fc3009e5912498 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:06:57.858659 31029 raft_consensus.cc:2258] T 56699fb289a74ae989fdb30ab5071e7b P d21e5a01a6bb4ceab3fc3009e5912498 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:06:57.858976 31028 tablet_replica.cc:330] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec: stopping tablet replica
I20260806 15:06:57.859048 31028 raft_consensus.cc:2229] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:06:57.859112 31028 raft_consensus.cc:2258] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:06:57.862231 31028 ts_tablet_manager.cc:1905] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:06:57.863806 31028 ts_tablet_manager.cc:1918] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:06:57.863881 31028 log.cc:1196] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec: Deleting WAL directory at /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028580791223-945-0/minicluster-data/ts-2/wal/wals/56699fb289a74ae989fdb30ab5071e7b
I20260806 15:06:57.864106 31028 ts_tablet_manager.cc:1939] T 56699fb289a74ae989fdb30ab5071e7b P 8baac205da574e5d9b44942f8a8a04ec: Deleting consensus metadata
I20260806 15:06:57.864460 30281 catalog_manager.cc:4841] TS 8baac205da574e5d9b44942f8a8a04ec (127.0.236.67:38457): tablet 56699fb289a74ae989fdb30ab5071e7b (table test-table [id=a515ad37b09d45a5b03bd6530aaa1db6]) successfully deleted
I20260806 15:06:57.872910 31030 tablet_replica.cc:330] T 56699fb289a74ae989fdb30ab5071e7b P 768d9ca1b6454fe199d1a35b48c38fba: stopping tablet replica
W20260806 15:06:57.872946 30282 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.236.65:35505 (error 108)
I20260806 15:06:57.873019 31030 raft_consensus.cc:2229] T 56699fb289a74ae989fdb30ab5071e7b P 768d9ca1b6454fe199d1a35b48c38fba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:06:57.873112 31030 raft_consensus.cc:2258] T 56699fb289a74ae989fdb30ab5071e7b P 768d9ca1b6454fe199d1a35b48c38fba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:06:57.873189 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 30597
I20260806 15:06:57.874476 30989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:57.873297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.65:35505 (local address 127.0.0.1:46388)
0806 15:06:57.874401 (+ 1104us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:57.874447 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.65:35505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1082}
W20260806 15:06:57.876684 30282 catalog_manager.cc:4584] TS d21e5a01a6bb4ceab3fc3009e5912498 (127.0.236.65:35505): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 56699fb289a74ae989fdb30ab5071e7b: Network error: Client connection negotiation failed: client connection to 127.0.236.65:35505: connect: Connection refused (error 111)
I20260806 15:06:57.910334 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 30787
I20260806 15:06:57.910404 30989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:57.910244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.66:37591 (local address 127.0.0.1:47582)
0806 15:06:57.910327 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0806 15:06:57.910363 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.66:37591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 15:06:57.925243 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 30264
2026-08-06T15:06:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (938 ms)
[----------] 6 tests from EncryptionPolicyTest (8582 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:06:57 dist-test-slave-tc2s krb5kdc[31036](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:06:57 dist-test-slave-tc2s krb5kdc[31036](info): set up 2 sockets
Aug 06 15:06:57 dist-test-slave-tc2s krb5kdc[31036](info): commencing operation
krb5kdc: starting...
W20260806 15:06:59.037590 945 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:06:59 dist-test-slave-tc2s krb5kdc[31036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:06:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:06:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 15:06:59.144225 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46409
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:06:59.222697 31051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:06:59.222887 31051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:06:59.222918 31051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:06:59.224013 31051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:06:59.224054 31051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:06:59.224074 31051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:06:59.224089 31051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:06:59.224105 31051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:06:59.225543 31051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46409
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:06:59.225772 31051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:06:59.225994 31051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:06:59.230403 31057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:59.230408 31059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:06:59.230408 31056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:06:59.230508 31051 server_base.cc:1034] running on GCE node
I20260806 15:06:59.230870 31051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:06:59.231130 31051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:06:59.232280 31051 hybrid_clock.cc:648] HybridClock initialized: now 1786028819232267 us; error 29 us; skew 500 ppm
Aug 06 15:06:59 dist-test-slave-tc2s krb5kdc[31036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028819, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:06:59.237128 31051 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 15:06:59.237522 31051 webserver.cc:460] Webserver started at http://127.0.0.1:42001/ using document root <none> and password file <none>
I20260806 15:06:59.237690 31051 fs_manager.cc:359] Metadata directory not provided
I20260806 15:06:59.237727 31051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:06:59.237815 31051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:06:59.238772 31051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "e3b3e87dd75d4ec9832f044f1b62675f"
format_stamp: "Formatted at 2026-08-06 15:06:59 on dist-test-slave-tc2s"
server_key: "ee1a5e706757fcd23f24d2d13ad040c7"
server_key_iv: "8d3d30e5e15f2bdbec11e72dab27eee2"
server_key_version: "encryptionkey@0"
I20260806 15:06:59.239068 31051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "e3b3e87dd75d4ec9832f044f1b62675f"
format_stamp: "Formatted at 2026-08-06 15:06:59 on dist-test-slave-tc2s"
server_key: "ee1a5e706757fcd23f24d2d13ad040c7"
server_key_iv: "8d3d30e5e15f2bdbec11e72dab27eee2"
server_key_version: "encryptionkey@0"
I20260806 15:06:59.240161 31051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:06:59.240779 31070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:06:59.240921 31051 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:06:59.241008 31051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "e3b3e87dd75d4ec9832f044f1b62675f"
format_stamp: "Formatted at 2026-08-06 15:06:59 on dist-test-slave-tc2s"
server_key: "ee1a5e706757fcd23f24d2d13ad040c7"
server_key_iv: "8d3d30e5e15f2bdbec11e72dab27eee2"
server_key_version: "encryptionkey@0"
I20260806 15:06:59.241067 31051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:06:59.247900 31051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:06:59.249480 31051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:06:59.249601 31051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:06:59.253371 31051 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46409
I20260806 15:06:59.253404 31140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46409 every 8 connection(s)
I20260806 15:06:59.253688 31051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:06:59.254482 31142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:06:59.256479 31142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3b3e87dd75d4ec9832f044f1b62675f: Bootstrap starting.
I20260806 15:06:59.256992 31142 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3b3e87dd75d4ec9832f044f1b62675f: Neither blocks nor log segments found. Creating new log.
I20260806 15:06:59.257238 31142 log.cc:824] T 00000000000000000000000000000000 P e3b3e87dd75d4ec9832f044f1b62675f: Log is configured to *not* fsync() on all Append() calls
I20260806 15:06:59.257844 31142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3b3e87dd75d4ec9832f044f1b62675f: No bootstrap required, opened a new log
I20260806 15:06:59.258062 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 31051
I20260806 15:06:59.258179 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:06:59.258363 945 external_mini_cluster.cc:1428] Setting key c430745a4d7dd6f8150ef8fb10fa6aed
I20260806 15:06:59.259881 31142 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3b3e87dd75d4ec9832f044f1b62675f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b3e87dd75d4ec9832f044f1b62675f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46409 } }
I20260806 15:06:59.260005 31142 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3b3e87dd75d4ec9832f044f1b62675f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:06:59.260026 31142 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3b3e87dd75d4ec9832f044f1b62675f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3b3e87dd75d4ec9832f044f1b62675f, State: Initialized, Role: FOLLOWER
I20260806 15:06:59.260128 31142 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3b3e87dd75d4ec9832f044f1b62675f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b3e87dd75d4ec9832f044f1b62675f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46409 } }
I20260806 15:06:59.260193 31142 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3b3e87dd75d4ec9832f044f1b62675f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:06:59.260221 31142 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3b3e87dd75d4ec9832f044f1b62675f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:06:59.260265 31142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3b3e87dd75d4ec9832f044f1b62675f [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:06:59.260900 31142 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3b3e87dd75d4ec9832f044f1b62675f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b3e87dd75d4ec9832f044f1b62675f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46409 } }
I20260806 15:06:59.261006 31142 leader_election.cc:302] T 00000000000000000000000000000000 P e3b3e87dd75d4ec9832f044f1b62675f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3b3e87dd75d4ec9832f044f1b62675f; no voters:
I20260806 15:06:59.261168 31142 leader_election.cc:288] T 00000000000000000000000000000000 P e3b3e87dd75d4ec9832f044f1b62675f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:06:59.261197 31149 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3b3e87dd75d4ec9832f044f1b62675f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:06:59.261312 31149 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3b3e87dd75d4ec9832f044f1b62675f [term 1 LEADER]: Becoming Leader. State: Replica: e3b3e87dd75d4ec9832f044f1b62675f, State: Running, Role: LEADER
I20260806 15:06:59.261391 31149 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3b3e87dd75d4ec9832f044f1b62675f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b3e87dd75d4ec9832f044f1b62675f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46409 } }
I20260806 15:06:59.261428 31142 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3b3e87dd75d4ec9832f044f1b62675f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:06:59.261699 31149 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3b3e87dd75d4ec9832f044f1b62675f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3b3e87dd75d4ec9832f044f1b62675f. Latest consensus state: current_term: 1 leader_uuid: "e3b3e87dd75d4ec9832f044f1b62675f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b3e87dd75d4ec9832f044f1b62675f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46409 } } }
I20260806 15:06:59.261765 31149 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3b3e87dd75d4ec9832f044f1b62675f [sys.catalog]: This master's current role is: LEADER
I20260806 15:06:59.261749 31150 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3b3e87dd75d4ec9832f044f1b62675f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3b3e87dd75d4ec9832f044f1b62675f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b3e87dd75d4ec9832f044f1b62675f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46409 } } }
I20260806 15:06:59.261842 31150 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3b3e87dd75d4ec9832f044f1b62675f [sys.catalog]: This master's current role is: LEADER
I20260806 15:06:59.262051 31157 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:06:59.262565 31157 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:06:59.264106 31157 catalog_manager.cc:1347] Generated new cluster ID: 9c9b0f23f53c48ef9dc4e8dd45901711
I20260806 15:06:59.264155 31157 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 15:06:59 dist-test-slave-tc2s krb5kdc[31036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 15:06:59.269011 31148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:59.258825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53257 (local address 127.0.0.1:46409)
0806 15:06:59.258993 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:59.259000 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:59.259024 (+ 24us) server_negotiation.cc:407] Connection header received
0806 15:06:59.260049 (+ 1025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:59.260060 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:59.260121 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:59.260255 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:59.261002 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:59.262314 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:59.263249 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:59.263810 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:59.263907 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:59.266076 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:59.266081 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:59.266081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:06:59.266091 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:59.267537 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:59.267893 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:59.267896 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:59.267896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:59.267985 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:59.268189 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:59.268190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:59.268191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:59.268401 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:59.268554 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:59.268681 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 15:06:59.268862 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
I20260806 15:06:59.269920 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:06:59.274276 31157 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:06:59.274928 31157 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:06:59.276583 31148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:06:59.271158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37979 (local address 127.0.0.1:46409)
0806 15:06:59.271235 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 15:06:59.271237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:06:59.271284 (+ 47us) server_negotiation.cc:407] Connection header received
0806 15:06:59.271347 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:06:59.271348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:06:59.271357 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:06:59.271400 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:06:59.271857 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:59.272311 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:59.272950 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:06:59.273165 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:06:59.273219 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:06:59.274336 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:06:59.274337 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:06:59.274337 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:06:59.274343 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:06:59.275559 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:59.275888 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:59.275889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:59.275890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:59.275949 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:06:59.276136 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:06:59.276137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:06:59.276137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:06:59.276234 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:06:59.276318 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 15:06:59.276451 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 15:06:59.276506 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 15:06:59.280378 31157 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3b3e87dd75d4ec9832f044f1b62675f: Generated new TSK 0
I20260806 15:06:59.280529 31157 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:06:59.287171 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 31051
W20260806 15:06:59.306213 31033 connection.cc:535] client connection to 127.0.0.1:46409 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46409): Connection reset by peer (error 104)
2026-08-06T15:06:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1371 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:06:59 dist-test-slave-tc2s krb5kdc[31184](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:06:59 dist-test-slave-tc2s krb5kdc[31184](info): set up 2 sockets
Aug 06 15:06:59 dist-test-slave-tc2s krb5kdc[31184](info): commencing operation
krb5kdc: starting...
W20260806 15:07:00.402951 945 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:07:00 dist-test-slave-tc2s krb5kdc[31184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:07:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:07:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 15:07:00.512002 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42809
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:07:00.589979 31200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:07:00.590163 31200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:07:00.590193 31200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:07:00.591282 31200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:07:00.591322 31200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:07:00.591341 31200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:07:00.591357 31200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:07:00.591372 31200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:07:00.592785 31200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42809
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:07:00.593014 31200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:07:00.593205 31200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:07:00.597601 31208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:00.597702 31200 server_base.cc:1034] running on GCE node
W20260806 15:07:00.597595 31211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:00.597595 31209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:00.598073 31200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:07:00.598341 31200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:07:00.599472 31200 hybrid_clock.cc:648] HybridClock initialized: now 1786028820599458 us; error 27 us; skew 500 ppm
Aug 06 15:07:00 dist-test-slave-tc2s krb5kdc[31184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028820, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:07:00.601727 31200 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 15:07:00.602151 31200 webserver.cc:460] Webserver started at http://127.0.0.1:42861/ using document root <none> and password file <none>
I20260806 15:07:00.602334 31200 fs_manager.cc:359] Metadata directory not provided
I20260806 15:07:00.602381 31200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:07:00.602499 31200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:07:00.603425 31200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "ab297190dafa45568520e2d13ad039ca"
format_stamp: "Formatted at 2026-08-06 15:07:00 on dist-test-slave-tc2s"
server_key: "aac4d30724f4c38d394eb4d0071a12dd"
server_key_iv: "3b01deff4b77e223637db7480397bb3d"
server_key_version: "encryptionkey@0"
I20260806 15:07:00.603726 31200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "ab297190dafa45568520e2d13ad039ca"
format_stamp: "Formatted at 2026-08-06 15:07:00 on dist-test-slave-tc2s"
server_key: "aac4d30724f4c38d394eb4d0071a12dd"
server_key_iv: "3b01deff4b77e223637db7480397bb3d"
server_key_version: "encryptionkey@0"
I20260806 15:07:00.604897 31200 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:07:00.605523 31220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:00.605683 31200 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:07:00.605763 31200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "ab297190dafa45568520e2d13ad039ca"
format_stamp: "Formatted at 2026-08-06 15:07:00 on dist-test-slave-tc2s"
server_key: "aac4d30724f4c38d394eb4d0071a12dd"
server_key_iv: "3b01deff4b77e223637db7480397bb3d"
server_key_version: "encryptionkey@0"
I20260806 15:07:00.605820 31200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:07:00.624899 31200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:07:00.626875 31200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:07:00.627009 31200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:07:00.630656 31200 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42809
I20260806 15:07:00.630697 31300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42809 every 8 connection(s)
I20260806 15:07:00.630992 31200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:07:00.631758 31302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:07:00.633854 31302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab297190dafa45568520e2d13ad039ca: Bootstrap starting.
I20260806 15:07:00.634436 31302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab297190dafa45568520e2d13ad039ca: Neither blocks nor log segments found. Creating new log.
I20260806 15:07:00.634697 31302 log.cc:824] T 00000000000000000000000000000000 P ab297190dafa45568520e2d13ad039ca: Log is configured to *not* fsync() on all Append() calls
I20260806 15:07:00.635326 31302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab297190dafa45568520e2d13ad039ca: No bootstrap required, opened a new log
I20260806 15:07:00.635937 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 31200
I20260806 15:07:00.636039 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:07:00.636201 945 external_mini_cluster.cc:1428] Setting key 80eef92d0edee9a713649efa2d3038f7
I20260806 15:07:00.637267 31302 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab297190dafa45568520e2d13ad039ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab297190dafa45568520e2d13ad039ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42809 } }
I20260806 15:07:00.637429 31302 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab297190dafa45568520e2d13ad039ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:07:00.637529 31302 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab297190dafa45568520e2d13ad039ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab297190dafa45568520e2d13ad039ca, State: Initialized, Role: FOLLOWER
I20260806 15:07:00.637666 31302 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab297190dafa45568520e2d13ad039ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab297190dafa45568520e2d13ad039ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42809 } }
I20260806 15:07:00.637748 31302 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab297190dafa45568520e2d13ad039ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:07:00.637787 31302 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab297190dafa45568520e2d13ad039ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:07:00.637835 31302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab297190dafa45568520e2d13ad039ca [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:07:00.638525 31302 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab297190dafa45568520e2d13ad039ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab297190dafa45568520e2d13ad039ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42809 } }
I20260806 15:07:00.638631 31302 leader_election.cc:302] T 00000000000000000000000000000000 P ab297190dafa45568520e2d13ad039ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab297190dafa45568520e2d13ad039ca; no voters:
I20260806 15:07:00.638789 31302 leader_election.cc:288] T 00000000000000000000000000000000 P ab297190dafa45568520e2d13ad039ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:07:00.638839 31309 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab297190dafa45568520e2d13ad039ca [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:07:00.638971 31309 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab297190dafa45568520e2d13ad039ca [term 1 LEADER]: Becoming Leader. State: Replica: ab297190dafa45568520e2d13ad039ca, State: Running, Role: LEADER
I20260806 15:07:00.639024 31302 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab297190dafa45568520e2d13ad039ca [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:07:00.639111 31309 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab297190dafa45568520e2d13ad039ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab297190dafa45568520e2d13ad039ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42809 } }
I20260806 15:07:00.639462 31309 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab297190dafa45568520e2d13ad039ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab297190dafa45568520e2d13ad039ca. Latest consensus state: current_term: 1 leader_uuid: "ab297190dafa45568520e2d13ad039ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab297190dafa45568520e2d13ad039ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42809 } } }
I20260806 15:07:00.639532 31309 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab297190dafa45568520e2d13ad039ca [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:00.639672 31310 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab297190dafa45568520e2d13ad039ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab297190dafa45568520e2d13ad039ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab297190dafa45568520e2d13ad039ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42809 } } }
I20260806 15:07:00.639717 31310 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab297190dafa45568520e2d13ad039ca [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:00.639811 31318 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:07:00.640503 31318 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:07:00.641880 31318 catalog_manager.cc:1347] Generated new cluster ID: cb140932c9444ef08aa282fa349d40b4
I20260806 15:07:00.642077 31318 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:07:00.642519 31307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:00.636652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59263 (local address 127.0.0.1:42809)
0806 15:07:00.636837 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:00.636846 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:00.636882 (+ 36us) server_negotiation.cc:407] Connection header received
0806 15:07:00.637241 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:00.637249 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:00.637319 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:00.637466 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:00.637694 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:00.639349 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:00.640465 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:00.641032 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:00.641133 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:00.641783 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:00.641786 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:00.641786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:07:00.642052 (+ 266us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:00.642089 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:07:00.642103 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:00.642191 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:00.642286 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 15:07:00.642312 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
I20260806 15:07:00.643461 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:07:00.648483 31307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:00.644989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51271 (local address 127.0.0.1:42809)
0806 15:07:00.645025 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:00.645027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:00.645112 (+ 85us) server_negotiation.cc:407] Connection header received
0806 15:07:00.645181 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:00.645182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:00.645191 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:00.645231 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:00.645364 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:00.646051 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:00.647027 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:00.647619 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:00.647678 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:00.648278 (+ 600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:00.648279 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:00.648280 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:07:00.648283 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:00.648296 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:07:00.648300 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:00.648346 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:00.648402 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 15:07:00.648418 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:07:00.656630 31318 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:07:00.657091 31318 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:07:00.664268 31318 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab297190dafa45568520e2d13ad039ca: Generated new TSK 0
I20260806 15:07:00.664415 31318 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:07:00.667024 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 31200
W20260806 15:07:00.677420 31181 connection.cc:535] client connection to 127.0.0.1:42809 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42809): Connection reset by peer (error 104)
2026-08-06T15:07:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1371 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:07:00 dist-test-slave-tc2s krb5kdc[31340](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:07:00 dist-test-slave-tc2s krb5kdc[31340](info): set up 2 sockets
Aug 06 15:07:00 dist-test-slave-tc2s krb5kdc[31340](info): commencing operation
krb5kdc: starting...
W20260806 15:07:01.776355 945 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:07:01 dist-test-slave-tc2s krb5kdc[31340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:07:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:07:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 15:07:01.883841 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36083
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36083
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:07:01.962188 31356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:07:01.962380 31356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:07:01.962401 31356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:07:01.963526 31356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:07:01.963567 31356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:07:01.963579 31356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:07:01.963593 31356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:07:01.963615 31356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:07:01.965080 31356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36083
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:07:01.965302 31356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:07:01.965495 31356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:07:01.970016 31367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:01.970063 31356 server_base.cc:1034] running on GCE node
W20260806 15:07:01.970019 31364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:01.970021 31363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:01.970386 31356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:07:01.970666 31356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:07:01.971807 31356 hybrid_clock.cc:648] HybridClock initialized: now 1786028821971793 us; error 29 us; skew 500 ppm
Aug 06 15:07:01 dist-test-slave-tc2s krb5kdc[31340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028821, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:07:01.974149 31356 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 15:07:01.974584 31356 webserver.cc:460] Webserver started at http://127.0.0.1:34841/ using document root <none> and password file <none>
I20260806 15:07:01.974781 31356 fs_manager.cc:359] Metadata directory not provided
I20260806 15:07:01.974828 31356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:07:01.974951 31356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:07:01.975924 31356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "363a62ef1f3a4e2cb6d380c56d69d756"
format_stamp: "Formatted at 2026-08-06 15:07:01 on dist-test-slave-tc2s"
server_key: "4e655b46471198faf400c1f3cd0121fa"
server_key_iv: "652cf20eb64b8a8f2481e3cc41d71a86"
server_key_version: "encryptionkey@0"
I20260806 15:07:01.976238 31356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "363a62ef1f3a4e2cb6d380c56d69d756"
format_stamp: "Formatted at 2026-08-06 15:07:01 on dist-test-slave-tc2s"
server_key: "4e655b46471198faf400c1f3cd0121fa"
server_key_iv: "652cf20eb64b8a8f2481e3cc41d71a86"
server_key_version: "encryptionkey@0"
I20260806 15:07:01.977439 31356 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:07:01.978103 31376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:01.978263 31356 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:01.978355 31356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "363a62ef1f3a4e2cb6d380c56d69d756"
format_stamp: "Formatted at 2026-08-06 15:07:01 on dist-test-slave-tc2s"
server_key: "4e655b46471198faf400c1f3cd0121fa"
server_key_iv: "652cf20eb64b8a8f2481e3cc41d71a86"
server_key_version: "encryptionkey@0"
I20260806 15:07:01.978416 31356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:07:01.981529 31356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:07:01.983170 31356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:07:01.983294 31356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:07:01.987020 31356 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36083
I20260806 15:07:01.987124 31449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36083 every 8 connection(s)
I20260806 15:07:01.987344 31356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:07:01.987571 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 31356
I20260806 15:07:01.987689 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:07:01.987870 945 external_mini_cluster.cc:1428] Setting key 644f716c6d3bb2d0de2aebd9e72b0bd0
I20260806 15:07:01.988055 31451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:07:01.990191 31456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:01.988350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56751 (local address 127.0.0.1:36083)
0806 15:07:01.988571 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:01.988578 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:01.988601 (+ 23us) server_negotiation.cc:407] Connection header received
0806 15:07:01.989004 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:01.989010 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:01.989041 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:01.989159 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:01.989800 (+ 641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:01.989803 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:01.989803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:07:01.989807 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:01.989854 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:07:01.989863 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:01.989951 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:01.990005 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 15:07:01.990042 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":117,"threads_started":1}
I20260806 15:07:01.991032 31451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 363a62ef1f3a4e2cb6d380c56d69d756: Bootstrap starting.
I20260806 15:07:01.991058 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:07:01.991595 31451 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 363a62ef1f3a4e2cb6d380c56d69d756: Neither blocks nor log segments found. Creating new log.
I20260806 15:07:01.991897 31451 log.cc:824] T 00000000000000000000000000000000 P 363a62ef1f3a4e2cb6d380c56d69d756: Log is configured to *not* fsync() on all Append() calls
I20260806 15:07:01.992547 31451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 363a62ef1f3a4e2cb6d380c56d69d756: No bootstrap required, opened a new log
I20260806 15:07:01.993155 31456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:01.992254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45673 (local address 127.0.0.1:36083)
0806 15:07:01.992293 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:01.992295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:01.992379 (+ 84us) server_negotiation.cc:407] Connection header received
0806 15:07:01.992440 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:01.992441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:01.992446 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:01.992490 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:01.992972 (+ 482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:01.992973 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:01.992974 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:07:01.992978 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:01.992989 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:07:01.992993 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:01.993038 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:01.993087 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 15:07:01.993099 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 15:07:01.994378 31451 raft_consensus.cc:348] T 00000000000000000000000000000000 P 363a62ef1f3a4e2cb6d380c56d69d756 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "363a62ef1f3a4e2cb6d380c56d69d756" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36083 } }
I20260806 15:07:01.994504 31451 raft_consensus.cc:374] T 00000000000000000000000000000000 P 363a62ef1f3a4e2cb6d380c56d69d756 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:07:01.994537 31451 raft_consensus.cc:729] T 00000000000000000000000000000000 P 363a62ef1f3a4e2cb6d380c56d69d756 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 363a62ef1f3a4e2cb6d380c56d69d756, State: Initialized, Role: FOLLOWER
I20260806 15:07:01.994644 31451 consensus_queue.cc:260] T 00000000000000000000000000000000 P 363a62ef1f3a4e2cb6d380c56d69d756 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "363a62ef1f3a4e2cb6d380c56d69d756" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36083 } }
I20260806 15:07:01.994714 31451 raft_consensus.cc:388] T 00000000000000000000000000000000 P 363a62ef1f3a4e2cb6d380c56d69d756 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:07:01.994745 31451 raft_consensus.cc:482] T 00000000000000000000000000000000 P 363a62ef1f3a4e2cb6d380c56d69d756 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:07:01.994794 31451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 363a62ef1f3a4e2cb6d380c56d69d756 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:07:01.995285 31451 raft_consensus.cc:504] T 00000000000000000000000000000000 P 363a62ef1f3a4e2cb6d380c56d69d756 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "363a62ef1f3a4e2cb6d380c56d69d756" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36083 } }
I20260806 15:07:01.995373 31451 leader_election.cc:302] T 00000000000000000000000000000000 P 363a62ef1f3a4e2cb6d380c56d69d756 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 363a62ef1f3a4e2cb6d380c56d69d756; no voters:
I20260806 15:07:01.995534 31451 leader_election.cc:288] T 00000000000000000000000000000000 P 363a62ef1f3a4e2cb6d380c56d69d756 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:07:01.995575 31465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 363a62ef1f3a4e2cb6d380c56d69d756 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:07:01.995714 31465 raft_consensus.cc:686] T 00000000000000000000000000000000 P 363a62ef1f3a4e2cb6d380c56d69d756 [term 1 LEADER]: Becoming Leader. State: Replica: 363a62ef1f3a4e2cb6d380c56d69d756, State: Running, Role: LEADER
I20260806 15:07:01.995755 31451 sys_catalog.cc:564] T 00000000000000000000000000000000 P 363a62ef1f3a4e2cb6d380c56d69d756 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:07:01.995802 31465 consensus_queue.cc:237] T 00000000000000000000000000000000 P 363a62ef1f3a4e2cb6d380c56d69d756 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "363a62ef1f3a4e2cb6d380c56d69d756" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36083 } }
I20260806 15:07:01.996196 31466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 363a62ef1f3a4e2cb6d380c56d69d756 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "363a62ef1f3a4e2cb6d380c56d69d756" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "363a62ef1f3a4e2cb6d380c56d69d756" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36083 } } }
I20260806 15:07:01.996202 31468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 363a62ef1f3a4e2cb6d380c56d69d756 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 363a62ef1f3a4e2cb6d380c56d69d756. Latest consensus state: current_term: 1 leader_uuid: "363a62ef1f3a4e2cb6d380c56d69d756" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "363a62ef1f3a4e2cb6d380c56d69d756" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36083 } } }
I20260806 15:07:01.996299 31466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 363a62ef1f3a4e2cb6d380c56d69d756 [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:01.996311 31468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 363a62ef1f3a4e2cb6d380c56d69d756 [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:01.996590 31479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:07:01.997208 31479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:07:01.998615 31479 catalog_manager.cc:1347] Generated new cluster ID: 24213d60149b462896c0c699fc205cbb
I20260806 15:07:01.998656 31479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:07:02.008157 31479 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:07:02.008296 31479 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:07:02.014042 31479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 363a62ef1f3a4e2cb6d380c56d69d756: Generated new TSK 0
I20260806 15:07:02.014194 31479 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:07:02.017783 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 31356
2026-08-06T15:07:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1349 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:07:02 dist-test-slave-tc2s krb5kdc[31499](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:07:02 dist-test-slave-tc2s krb5kdc[31499](info): set up 2 sockets
Aug 06 15:07:02 dist-test-slave-tc2s krb5kdc[31499](info): commencing operation
krb5kdc: starting...
W20260806 15:07:03.129045 945 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:07:03 dist-test-slave-tc2s krb5kdc[31499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:07:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:07:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 15:07:03.240542 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44835
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:07:03.319252 31514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:07:03.319449 31514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:07:03.319473 31514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:07:03.320560 31514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:07:03.320598 31514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:07:03.320611 31514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:07:03.320621 31514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:07:03.320636 31514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:07:03.322045 31514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:07:03.322223 31514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:07:03.322397 31514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:07:03.326853 31528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:03.326858 31527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:03.326912 31530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:03.326930 31514 server_base.cc:1034] running on GCE node
I20260806 15:07:03.327292 31514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:07:03.327539 31514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:07:03.328676 31514 hybrid_clock.cc:648] HybridClock initialized: now 1786028823328663 us; error 29 us; skew 500 ppm
Aug 06 15:07:03 dist-test-slave-tc2s krb5kdc[31499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028823, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:07:03.331012 31514 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 15:07:03.331388 31514 webserver.cc:460] Webserver started at http://127.0.0.1:45089/ using document root <none> and password file <none>
I20260806 15:07:03.331602 31514 fs_manager.cc:359] Metadata directory not provided
I20260806 15:07:03.331650 31514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:07:03.331769 31514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:07:03.332762 31514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "540f9cbe56864fab8cf3656e7db7613a"
format_stamp: "Formatted at 2026-08-06 15:07:03 on dist-test-slave-tc2s"
server_key: "5689b65941bead6adea99fa8459b4cf7"
server_key_iv: "314b0b01951a07474a0dca6f3f351272"
server_key_version: "encryptionkey@0"
I20260806 15:07:03.333078 31514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "540f9cbe56864fab8cf3656e7db7613a"
format_stamp: "Formatted at 2026-08-06 15:07:03 on dist-test-slave-tc2s"
server_key: "5689b65941bead6adea99fa8459b4cf7"
server_key_iv: "314b0b01951a07474a0dca6f3f351272"
server_key_version: "encryptionkey@0"
I20260806 15:07:03.334324 31514 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:07:03.334952 31539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:03.335099 31514 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:07:03.335183 31514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "540f9cbe56864fab8cf3656e7db7613a"
format_stamp: "Formatted at 2026-08-06 15:07:03 on dist-test-slave-tc2s"
server_key: "5689b65941bead6adea99fa8459b4cf7"
server_key_iv: "314b0b01951a07474a0dca6f3f351272"
server_key_version: "encryptionkey@0"
I20260806 15:07:03.335243 31514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:07:03.346745 31514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:07:03.348316 31514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:07:03.348443 31514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:07:03.352245 31514 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44835
I20260806 15:07:03.352285 31613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44835 every 8 connection(s)
I20260806 15:07:03.352579 31514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:07:03.353377 31614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:07:03.354532 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 31514
I20260806 15:07:03.354646 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:07:03.354809 945 external_mini_cluster.cc:1428] Setting key 7ca39c736b948740f483b5826fb166dd
I20260806 15:07:03.355882 31614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 540f9cbe56864fab8cf3656e7db7613a: Bootstrap starting.
I20260806 15:07:03.356527 31614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 540f9cbe56864fab8cf3656e7db7613a: Neither blocks nor log segments found. Creating new log.
I20260806 15:07:03.356835 31614 log.cc:824] T 00000000000000000000000000000000 P 540f9cbe56864fab8cf3656e7db7613a: Log is configured to *not* fsync() on all Append() calls
I20260806 15:07:03.357560 31614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 540f9cbe56864fab8cf3656e7db7613a: No bootstrap required, opened a new log
I20260806 15:07:03.359723 31614 raft_consensus.cc:348] T 00000000000000000000000000000000 P 540f9cbe56864fab8cf3656e7db7613a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540f9cbe56864fab8cf3656e7db7613a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44835 } }
I20260806 15:07:03.359930 31614 raft_consensus.cc:374] T 00000000000000000000000000000000 P 540f9cbe56864fab8cf3656e7db7613a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:07:03.359988 31614 raft_consensus.cc:729] T 00000000000000000000000000000000 P 540f9cbe56864fab8cf3656e7db7613a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 540f9cbe56864fab8cf3656e7db7613a, State: Initialized, Role: FOLLOWER
I20260806 15:07:03.360149 31614 consensus_queue.cc:260] T 00000000000000000000000000000000 P 540f9cbe56864fab8cf3656e7db7613a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540f9cbe56864fab8cf3656e7db7613a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44835 } }
I20260806 15:07:03.360260 31614 raft_consensus.cc:388] T 00000000000000000000000000000000 P 540f9cbe56864fab8cf3656e7db7613a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:07:03.360306 31614 raft_consensus.cc:482] T 00000000000000000000000000000000 P 540f9cbe56864fab8cf3656e7db7613a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:07:03.360383 31614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 540f9cbe56864fab8cf3656e7db7613a [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:07:03.361045 31614 raft_consensus.cc:504] T 00000000000000000000000000000000 P 540f9cbe56864fab8cf3656e7db7613a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540f9cbe56864fab8cf3656e7db7613a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44835 } }
I20260806 15:07:03.361153 31614 leader_election.cc:302] T 00000000000000000000000000000000 P 540f9cbe56864fab8cf3656e7db7613a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 540f9cbe56864fab8cf3656e7db7613a; no voters:
I20260806 15:07:03.361322 31614 leader_election.cc:288] T 00000000000000000000000000000000 P 540f9cbe56864fab8cf3656e7db7613a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:07:03.361421 31620 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 540f9cbe56864fab8cf3656e7db7613a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:07:03.361621 31620 raft_consensus.cc:686] T 00000000000000000000000000000000 P 540f9cbe56864fab8cf3656e7db7613a [term 1 LEADER]: Becoming Leader. State: Replica: 540f9cbe56864fab8cf3656e7db7613a, State: Running, Role: LEADER
I20260806 15:07:03.361663 31614 sys_catalog.cc:564] T 00000000000000000000000000000000 P 540f9cbe56864fab8cf3656e7db7613a [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 15:07:03 dist-test-slave-tc2s krb5kdc[31499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 15:07:03.362018 31620 consensus_queue.cc:237] T 00000000000000000000000000000000 P 540f9cbe56864fab8cf3656e7db7613a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540f9cbe56864fab8cf3656e7db7613a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44835 } }
I20260806 15:07:03.362394 31622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 540f9cbe56864fab8cf3656e7db7613a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "540f9cbe56864fab8cf3656e7db7613a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540f9cbe56864fab8cf3656e7db7613a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44835 } } }
I20260806 15:07:03.362473 31622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 540f9cbe56864fab8cf3656e7db7613a [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:03.362428 31621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 540f9cbe56864fab8cf3656e7db7613a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 540f9cbe56864fab8cf3656e7db7613a. Latest consensus state: current_term: 1 leader_uuid: "540f9cbe56864fab8cf3656e7db7613a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540f9cbe56864fab8cf3656e7db7613a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44835 } } }
I20260806 15:07:03.362516 31621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 540f9cbe56864fab8cf3656e7db7613a [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:03.362668 31633 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:07:03.363410 31633 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:07:03.364880 31633 catalog_manager.cc:1347] Generated new cluster ID: 96efb5beadff4e03ba693722217225ec
I20260806 15:07:03.364936 31633 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:07:03.366066 31618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:03.355251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35835 (local address 127.0.0.1:44835)
0806 15:07:03.355437 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:03.355446 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:03.355481 (+ 35us) server_negotiation.cc:407] Connection header received
0806 15:07:03.355963 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:03.355971 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:03.356029 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:03.356172 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:03.356732 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:03.358031 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:03.358803 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:03.359331 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:03.359430 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:03.362462 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:03.362467 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:03.362467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:03.362479 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:03.364436 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:03.364862 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:03.364865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:03.364866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:03.364984 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:03.365193 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:03.365194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:03.365194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:03.365472 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:03.365615 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:03.365722 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 15:07:03.365845 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
I20260806 15:07:03.370779 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:07:03.378706 31618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:03.372241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50945 (local address 127.0.0.1:44835)
0806 15:07:03.372277 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:03.372279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:03.372309 (+ 30us) server_negotiation.cc:407] Connection header received
0806 15:07:03.372327 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:03.372328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:03.372336 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:03.372389 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:03.372928 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:03.373649 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:03.374602 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:03.374942 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:03.375010 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:03.376207 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:03.376208 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:03.376208 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:03.376215 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:03.377653 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:03.378021 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:03.378022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:03.378022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:03.378081 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:03.378243 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:03.378243 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:03.378243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:03.378362 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:03.378449 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:03.378544 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 15:07:03.378617 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:07:03.382318 31633 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:07:03.382799 31633 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:07:03.385427 31633 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 540f9cbe56864fab8cf3656e7db7613a: Generated new TSK 0
I20260806 15:07:03.385582 31633 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:07:03.391790 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 31514
2026-08-06T15:07:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1384 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:07:03 dist-test-slave-tc2s krb5kdc[31656](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:07:03 dist-test-slave-tc2s krb5kdc[31656](info): set up 2 sockets
Aug 06 15:07:03 dist-test-slave-tc2s krb5kdc[31656](info): commencing operation
krb5kdc: starting...
W20260806 15:07:04.516867 945 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:07:04 dist-test-slave-tc2s krb5kdc[31656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:07:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:07:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 15:07:04.625154 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42649
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:07:04.703563 31678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:07:04.703747 31678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:07:04.703768 31678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:07:04.704864 31678 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:07:04.704902 31678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:07:04.704917 31678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:07:04.704941 31678 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:07:04.704962 31678 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:07:04.706461 31678 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42649
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:07:04.706673 31678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:07:04.706861 31678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:07:04.711277 31686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:04.711319 31678 server_base.cc:1034] running on GCE node
W20260806 15:07:04.711308 31688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:04.711288 31685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:04.711666 31678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:07:04.711930 31678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:07:04.713063 31678 hybrid_clock.cc:648] HybridClock initialized: now 1786028824713050 us; error 25 us; skew 500 ppm
Aug 06 15:07:04 dist-test-slave-tc2s krb5kdc[31656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028824, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:07:04.719013 31678 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 15:07:04.719403 31678 webserver.cc:460] Webserver started at http://127.0.0.1:34379/ using document root <none> and password file <none>
I20260806 15:07:04.719589 31678 fs_manager.cc:359] Metadata directory not provided
I20260806 15:07:04.719636 31678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:07:04.719756 31678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:07:04.720731 31678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "1e0a4c83c89d4a988596a4b81cea3997"
format_stamp: "Formatted at 2026-08-06 15:07:04 on dist-test-slave-tc2s"
server_key: "ffebe330a69c87c8d0b5db2b5cf33d70"
server_key_iv: "fd9f0bf5d1dfc0be3e950933e7317679"
server_key_version: "encryptionkey@0"
I20260806 15:07:04.721047 31678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "1e0a4c83c89d4a988596a4b81cea3997"
format_stamp: "Formatted at 2026-08-06 15:07:04 on dist-test-slave-tc2s"
server_key: "ffebe330a69c87c8d0b5db2b5cf33d70"
server_key_iv: "fd9f0bf5d1dfc0be3e950933e7317679"
server_key_version: "encryptionkey@0"
I20260806 15:07:04.722262 31678 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:07:04.722892 31696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:04.723042 31678 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:07:04.723124 31678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "1e0a4c83c89d4a988596a4b81cea3997"
format_stamp: "Formatted at 2026-08-06 15:07:04 on dist-test-slave-tc2s"
server_key: "ffebe330a69c87c8d0b5db2b5cf33d70"
server_key_iv: "fd9f0bf5d1dfc0be3e950933e7317679"
server_key_version: "encryptionkey@0"
I20260806 15:07:04.723178 31678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:07:04.763257 31678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:07:04.765282 31678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:07:04.765410 31678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:07:04.769124 31678 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42649
I20260806 15:07:04.769170 31773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42649 every 8 connection(s)
I20260806 15:07:04.769464 31678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:07:04.770299 31776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:07:04.772432 31776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e0a4c83c89d4a988596a4b81cea3997: Bootstrap starting.
I20260806 15:07:04.772938 31776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e0a4c83c89d4a988596a4b81cea3997: Neither blocks nor log segments found. Creating new log.
I20260806 15:07:04.773169 31776 log.cc:824] T 00000000000000000000000000000000 P 1e0a4c83c89d4a988596a4b81cea3997: Log is configured to *not* fsync() on all Append() calls
I20260806 15:07:04.773761 31776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e0a4c83c89d4a988596a4b81cea3997: No bootstrap required, opened a new log
I20260806 15:07:04.775264 31776 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e0a4c83c89d4a988596a4b81cea3997 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e0a4c83c89d4a988596a4b81cea3997" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42649 } }
I20260806 15:07:04.775403 31776 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e0a4c83c89d4a988596a4b81cea3997 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:07:04.775430 31776 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e0a4c83c89d4a988596a4b81cea3997 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e0a4c83c89d4a988596a4b81cea3997, State: Initialized, Role: FOLLOWER
I20260806 15:07:04.775533 31776 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e0a4c83c89d4a988596a4b81cea3997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e0a4c83c89d4a988596a4b81cea3997" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42649 } }
I20260806 15:07:04.775607 31776 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e0a4c83c89d4a988596a4b81cea3997 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:07:04.775635 31776 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e0a4c83c89d4a988596a4b81cea3997 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:07:04.775667 31776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e0a4c83c89d4a988596a4b81cea3997 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:07:04.776099 31776 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e0a4c83c89d4a988596a4b81cea3997 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e0a4c83c89d4a988596a4b81cea3997" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42649 } }
I20260806 15:07:04.776180 31776 leader_election.cc:302] T 00000000000000000000000000000000 P 1e0a4c83c89d4a988596a4b81cea3997 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e0a4c83c89d4a988596a4b81cea3997; no voters:
I20260806 15:07:04.776304 31776 leader_election.cc:288] T 00000000000000000000000000000000 P 1e0a4c83c89d4a988596a4b81cea3997 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:07:04.776346 31780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e0a4c83c89d4a988596a4b81cea3997 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:07:04.776495 31780 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e0a4c83c89d4a988596a4b81cea3997 [term 1 LEADER]: Becoming Leader. State: Replica: 1e0a4c83c89d4a988596a4b81cea3997, State: Running, Role: LEADER
I20260806 15:07:04.776520 31776 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e0a4c83c89d4a988596a4b81cea3997 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:07:04.776597 31780 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e0a4c83c89d4a988596a4b81cea3997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e0a4c83c89d4a988596a4b81cea3997" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42649 } }
I20260806 15:07:04.776916 31781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e0a4c83c89d4a988596a4b81cea3997 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e0a4c83c89d4a988596a4b81cea3997" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e0a4c83c89d4a988596a4b81cea3997" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42649 } } }
I20260806 15:07:04.776921 31782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e0a4c83c89d4a988596a4b81cea3997 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e0a4c83c89d4a988596a4b81cea3997. Latest consensus state: current_term: 1 leader_uuid: "1e0a4c83c89d4a988596a4b81cea3997" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e0a4c83c89d4a988596a4b81cea3997" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42649 } } }
I20260806 15:07:04.777038 31781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e0a4c83c89d4a988596a4b81cea3997 [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:04.777047 31782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e0a4c83c89d4a988596a4b81cea3997 [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:04.777261 31784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:07:04.777767 31784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:07:04.779234 31784 catalog_manager.cc:1347] Generated new cluster ID: dad13fb53af242d885a060213ed3401f
I20260806 15:07:04.779289 31784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:07:04.779343 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 31678
I20260806 15:07:04.779486 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:07:04.779644 945 external_mini_cluster.cc:1428] Setting key d5c1c91a8cb6ade2fa9ff10176d9175a
I20260806 15:07:04.785254 31807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:04.780037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60863 (local address 127.0.0.1:42649)
0806 15:07:04.780186 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:04.780195 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:04.780377 (+ 182us) server_negotiation.cc:407] Connection header received
0806 15:07:04.780516 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:04.780523 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:04.780573 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:04.780704 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:04.780903 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:04.782270 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:04.783343 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:04.783912 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:04.783998 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:04.784642 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:04.784645 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:04.784645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:07:04.784828 (+ 183us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:04.784863 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:07:04.784876 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:04.784929 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:04.785003 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 15:07:04.785058 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":67,"threads_started":1}
I20260806 15:07:04.786082 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:07:04.790524 31807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:04.787413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36379 (local address 127.0.0.1:42649)
0806 15:07:04.787441 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:04.787442 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:04.787567 (+ 125us) server_negotiation.cc:407] Connection header received
0806 15:07:04.787627 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:04.787627 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:04.787636 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:04.787673 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:04.787794 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:04.788460 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:04.789402 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:04.789684 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:04.789737 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:04.790316 (+ 579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:04.790317 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:04.790317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:07:04.790321 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:04.790334 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:07:04.790338 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:04.790383 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:04.790434 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 15:07:04.790456 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260806 15:07:04.791934 31784 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:07:04.792392 31784 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:07:04.797344 31784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e0a4c83c89d4a988596a4b81cea3997: Generated new TSK 0
I20260806 15:07:04.797495 31784 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:07:04.799351 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 31678
2026-08-06T15:07:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1401 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:07:04 dist-test-slave-tc2s krb5kdc[31818](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:07:04 dist-test-slave-tc2s krb5kdc[31818](info): set up 2 sockets
Aug 06 15:07:04 dist-test-slave-tc2s krb5kdc[31818](info): commencing operation
krb5kdc: starting...
W20260806 15:07:05.918488 945 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:07:05 dist-test-slave-tc2s krb5kdc[31818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:07:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:07:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 15:07:06.026538 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44205
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:07:06.105253 31839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:07:06.105460 31839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:07:06.105492 31839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:07:06.106652 31839 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:07:06.106691 31839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:07:06.106711 31839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:07:06.106727 31839 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:07:06.106743 31839 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:07:06.108289 31839 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:07:06.108536 31839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:07:06.108734 31839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:07:06.113220 31847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:06.113309 31839 server_base.cc:1034] running on GCE node
W20260806 15:07:06.113255 31850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:06.113219 31848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:06.113680 31839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:07:06.113955 31839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:07:06.115091 31839 hybrid_clock.cc:648] HybridClock initialized: now 1786028826115079 us; error 29 us; skew 500 ppm
Aug 06 15:07:06 dist-test-slave-tc2s krb5kdc[31818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028826, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:07:06.117419 31839 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 15:07:06.117805 31839 webserver.cc:460] Webserver started at http://127.0.0.1:34659/ using document root <none> and password file <none>
I20260806 15:07:06.118036 31839 fs_manager.cc:359] Metadata directory not provided
I20260806 15:07:06.118081 31839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:07:06.118196 31839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:07:06.119171 31839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "5979d76060de4dff8f0caca62599722c"
format_stamp: "Formatted at 2026-08-06 15:07:06 on dist-test-slave-tc2s"
server_key: "962a8137980d1ea33a72640771f7779d"
server_key_iv: "634d7debc02173a16a39803ee4d11ebd"
server_key_version: "encryptionkey@0"
I20260806 15:07:06.119483 31839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "5979d76060de4dff8f0caca62599722c"
format_stamp: "Formatted at 2026-08-06 15:07:06 on dist-test-slave-tc2s"
server_key: "962a8137980d1ea33a72640771f7779d"
server_key_iv: "634d7debc02173a16a39803ee4d11ebd"
server_key_version: "encryptionkey@0"
I20260806 15:07:06.120694 31839 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:07:06.121297 31864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:06.121451 31839 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:07:06.121534 31839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "5979d76060de4dff8f0caca62599722c"
format_stamp: "Formatted at 2026-08-06 15:07:06 on dist-test-slave-tc2s"
server_key: "962a8137980d1ea33a72640771f7779d"
server_key_iv: "634d7debc02173a16a39803ee4d11ebd"
server_key_version: "encryptionkey@0"
I20260806 15:07:06.121587 31839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:07:06.124868 31839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:07:06.126605 31839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:07:06.126722 31839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:07:06.130431 31839 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44205
I20260806 15:07:06.130460 31936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44205 every 8 connection(s)
I20260806 15:07:06.130733 31839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:07:06.131650 31937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:07:06.133733 31937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5979d76060de4dff8f0caca62599722c: Bootstrap starting.
I20260806 15:07:06.134300 31937 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5979d76060de4dff8f0caca62599722c: Neither blocks nor log segments found. Creating new log.
I20260806 15:07:06.134573 31937 log.cc:824] T 00000000000000000000000000000000 P 5979d76060de4dff8f0caca62599722c: Log is configured to *not* fsync() on all Append() calls
I20260806 15:07:06.135203 31937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5979d76060de4dff8f0caca62599722c: No bootstrap required, opened a new log
I20260806 15:07:06.136724 31937 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5979d76060de4dff8f0caca62599722c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5979d76060de4dff8f0caca62599722c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44205 } }
I20260806 15:07:06.136852 31937 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5979d76060de4dff8f0caca62599722c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:07:06.136884 31937 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5979d76060de4dff8f0caca62599722c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5979d76060de4dff8f0caca62599722c, State: Initialized, Role: FOLLOWER
I20260806 15:07:06.137023 31937 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5979d76060de4dff8f0caca62599722c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5979d76060de4dff8f0caca62599722c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44205 } }
I20260806 15:07:06.137097 31937 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5979d76060de4dff8f0caca62599722c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:07:06.137133 31937 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5979d76060de4dff8f0caca62599722c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:07:06.137179 31937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5979d76060de4dff8f0caca62599722c [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:07:06.137670 31937 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5979d76060de4dff8f0caca62599722c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5979d76060de4dff8f0caca62599722c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44205 } }
I20260806 15:07:06.137760 31937 leader_election.cc:302] T 00000000000000000000000000000000 P 5979d76060de4dff8f0caca62599722c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5979d76060de4dff8f0caca62599722c; no voters:
I20260806 15:07:06.137928 31937 leader_election.cc:288] T 00000000000000000000000000000000 P 5979d76060de4dff8f0caca62599722c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:07:06.137972 31951 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5979d76060de4dff8f0caca62599722c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:07:06.138095 31951 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5979d76060de4dff8f0caca62599722c [term 1 LEADER]: Becoming Leader. State: Replica: 5979d76060de4dff8f0caca62599722c, State: Running, Role: LEADER
I20260806 15:07:06.138178 31937 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5979d76060de4dff8f0caca62599722c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:07:06.138212 31951 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5979d76060de4dff8f0caca62599722c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5979d76060de4dff8f0caca62599722c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44205 } }
I20260806 15:07:06.138554 31953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5979d76060de4dff8f0caca62599722c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5979d76060de4dff8f0caca62599722c. Latest consensus state: current_term: 1 leader_uuid: "5979d76060de4dff8f0caca62599722c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5979d76060de4dff8f0caca62599722c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44205 } } }
I20260806 15:07:06.138633 31953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5979d76060de4dff8f0caca62599722c [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:06.138675 31952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5979d76060de4dff8f0caca62599722c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5979d76060de4dff8f0caca62599722c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5979d76060de4dff8f0caca62599722c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44205 } } }
I20260806 15:07:06.138725 31952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5979d76060de4dff8f0caca62599722c [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:06.139101 31959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:07:06.139555 31959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:07:06.140417 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 31839
I20260806 15:07:06.140522 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:07:06.140684 945 external_mini_cluster.cc:1428] Setting key bc00ab1db227348910584e2d5bdd5db7
I20260806 15:07:06.141139 31959 catalog_manager.cc:1347] Generated new cluster ID: b3bf774ef6d341bdbda4791c1e2455c0
I20260806 15:07:06.141185 31959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:07:06.142725 31974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:06.141119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35115 (local address 127.0.0.1:44205)
0806 15:07:06.141425 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:06.141433 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:06.141460 (+ 27us) server_negotiation.cc:407] Connection header received
0806 15:07:06.141560 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:06.141565 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:06.141593 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:06.141705 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:06.142341 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:06.142344 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:06.142344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:07:06.142347 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:06.142380 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:07:06.142392 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:06.142452 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:06.142520 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 15:07:06.142556 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":65,"threads_started":1}
I20260806 15:07:06.143596 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:07:06.146042 31974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:06.145102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56937 (local address 127.0.0.1:44205)
0806 15:07:06.145139 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:06.145141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:06.145148 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:07:06.145227 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:06.145228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:06.145233 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:06.145285 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:06.145819 (+ 534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:06.145820 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:06.145821 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:07:06.145825 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:06.145840 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:07:06.145845 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:06.145925 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:06.145970 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 15:07:06.145984 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 15:07:06.154659 31959 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:07:06.154798 31959 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:07:06.166477 31959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5979d76060de4dff8f0caca62599722c: Generated new TSK 0
I20260806 15:07:06.166631 31959 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:07:06.173151 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 31839
2026-08-06T15:07:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1369 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:07:06 dist-test-slave-tc2s krb5kdc[31994](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:07:06 dist-test-slave-tc2s krb5kdc[31994](info): set up 2 sockets
Aug 06 15:07:06 dist-test-slave-tc2s krb5kdc[31994](info): commencing operation
krb5kdc: starting...
W20260806 15:07:07.274982 945 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:07:07 dist-test-slave-tc2s krb5kdc[31994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:07:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:07:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 15:07:07.383921 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45827
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45827
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:07:07.464514 32011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:07:07.464707 32011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:07:07.464730 32011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:07:07.465853 32011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:07:07.465912 32011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:07:07.465950 32011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:07:07.465967 32011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:07:07.465991 32011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:07:07.467447 32011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45827
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:07:07.467657 32011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:07:07.467849 32011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:07:07.472245 32022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:07.472379 32011 server_base.cc:1034] running on GCE node
W20260806 15:07:07.472435 32018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:07.472246 32020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:07.472680 32011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:07:07.472944 32011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:07:07.474081 32011 hybrid_clock.cc:648] HybridClock initialized: now 1786028827474067 us; error 26 us; skew 500 ppm
Aug 06 15:07:07 dist-test-slave-tc2s krb5kdc[31994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028827, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:07:07.476270 32011 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 15:07:07.476668 32011 webserver.cc:460] Webserver started at http://127.0.0.1:44679/ using document root <none> and password file <none>
I20260806 15:07:07.476852 32011 fs_manager.cc:359] Metadata directory not provided
I20260806 15:07:07.476898 32011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:07:07.477017 32011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:07:07.478025 32011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "1bfbebae1aed4b0f8cc021d8b0af0f92"
format_stamp: "Formatted at 2026-08-06 15:07:07 on dist-test-slave-tc2s"
server_key: "742f14807cac404c4c59e84c65efbcf8"
server_key_iv: "93c9ff75bb2ef5dc4de93832b3ba977a"
server_key_version: "encryptionkey@0"
I20260806 15:07:07.478333 32011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "1bfbebae1aed4b0f8cc021d8b0af0f92"
format_stamp: "Formatted at 2026-08-06 15:07:07 on dist-test-slave-tc2s"
server_key: "742f14807cac404c4c59e84c65efbcf8"
server_key_iv: "93c9ff75bb2ef5dc4de93832b3ba977a"
server_key_version: "encryptionkey@0"
I20260806 15:07:07.479506 32011 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:07:07.480111 32032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:07.480260 32011 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:07:07.480350 32011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "1bfbebae1aed4b0f8cc021d8b0af0f92"
format_stamp: "Formatted at 2026-08-06 15:07:07 on dist-test-slave-tc2s"
server_key: "742f14807cac404c4c59e84c65efbcf8"
server_key_iv: "93c9ff75bb2ef5dc4de93832b3ba977a"
server_key_version: "encryptionkey@0"
I20260806 15:07:07.480407 32011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:07:07.495357 32011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:07:07.497073 32011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:07:07.497205 32011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:07:07.500995 32011 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45827
I20260806 15:07:07.501031 32107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45827 every 8 connection(s)
I20260806 15:07:07.501348 32011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:07:07.502209 32110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:07:07.504384 32110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bfbebae1aed4b0f8cc021d8b0af0f92: Bootstrap starting.
I20260806 15:07:07.504917 32110 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1bfbebae1aed4b0f8cc021d8b0af0f92: Neither blocks nor log segments found. Creating new log.
I20260806 15:07:07.505151 32110 log.cc:824] T 00000000000000000000000000000000 P 1bfbebae1aed4b0f8cc021d8b0af0f92: Log is configured to *not* fsync() on all Append() calls
I20260806 15:07:07.505765 32110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bfbebae1aed4b0f8cc021d8b0af0f92: No bootstrap required, opened a new log
I20260806 15:07:07.507287 32110 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1bfbebae1aed4b0f8cc021d8b0af0f92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bfbebae1aed4b0f8cc021d8b0af0f92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45827 } }
I20260806 15:07:07.507421 32110 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1bfbebae1aed4b0f8cc021d8b0af0f92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:07:07.507460 32110 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1bfbebae1aed4b0f8cc021d8b0af0f92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bfbebae1aed4b0f8cc021d8b0af0f92, State: Initialized, Role: FOLLOWER
I20260806 15:07:07.507577 32110 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1bfbebae1aed4b0f8cc021d8b0af0f92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bfbebae1aed4b0f8cc021d8b0af0f92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45827 } }
I20260806 15:07:07.507650 32110 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1bfbebae1aed4b0f8cc021d8b0af0f92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:07:07.507678 32110 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1bfbebae1aed4b0f8cc021d8b0af0f92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:07:07.507710 32110 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1bfbebae1aed4b0f8cc021d8b0af0f92 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:07:07.508158 32110 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1bfbebae1aed4b0f8cc021d8b0af0f92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bfbebae1aed4b0f8cc021d8b0af0f92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45827 } }
I20260806 15:07:07.508246 32110 leader_election.cc:302] T 00000000000000000000000000000000 P 1bfbebae1aed4b0f8cc021d8b0af0f92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1bfbebae1aed4b0f8cc021d8b0af0f92; no voters:
I20260806 15:07:07.508299 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 32011
I20260806 15:07:07.508395 32110 leader_election.cc:288] T 00000000000000000000000000000000 P 1bfbebae1aed4b0f8cc021d8b0af0f92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:07:07.508417 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:07:07.508447 32115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1bfbebae1aed4b0f8cc021d8b0af0f92 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:07:07.508613 945 external_mini_cluster.cc:1428] Setting key 5e053eaa56866a666673c2664fc596d2
I20260806 15:07:07.508613 32115 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1bfbebae1aed4b0f8cc021d8b0af0f92 [term 1 LEADER]: Becoming Leader. State: Replica: 1bfbebae1aed4b0f8cc021d8b0af0f92, State: Running, Role: LEADER
I20260806 15:07:07.508664 32110 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1bfbebae1aed4b0f8cc021d8b0af0f92 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:07:07.508694 32115 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1bfbebae1aed4b0f8cc021d8b0af0f92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bfbebae1aed4b0f8cc021d8b0af0f92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45827 } }
I20260806 15:07:07.509370 32116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bfbebae1aed4b0f8cc021d8b0af0f92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1bfbebae1aed4b0f8cc021d8b0af0f92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bfbebae1aed4b0f8cc021d8b0af0f92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45827 } } }
I20260806 15:07:07.509392 32117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bfbebae1aed4b0f8cc021d8b0af0f92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1bfbebae1aed4b0f8cc021d8b0af0f92. Latest consensus state: current_term: 1 leader_uuid: "1bfbebae1aed4b0f8cc021d8b0af0f92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bfbebae1aed4b0f8cc021d8b0af0f92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45827 } } }
I20260806 15:07:07.509506 32116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bfbebae1aed4b0f8cc021d8b0af0f92 [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:07.509526 32117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bfbebae1aed4b0f8cc021d8b0af0f92 [sys.catalog]: This master's current role is: LEADER
W20260806 15:07:07.511041 32141 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1bfbebae1aed4b0f8cc021d8b0af0f92: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:07:07.511097 32141 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:07:07.511152 32130 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:07:07.511315 32130 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:07:07.513468 32130 catalog_manager.cc:1347] Generated new cluster ID: 1129b1251db24ec6a00c91dbba2b7401
I20260806 15:07:07.513521 32130 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 15:07:07 dist-test-slave-tc2s krb5kdc[31994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 15:07:07.520171 32129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:07.509356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:47601 (local address 127.0.0.1:45827)
0806 15:07:07.509528 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:07.509537 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:07.509785 (+ 248us) server_negotiation.cc:407] Connection header received
0806 15:07:07.509944 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:07.509951 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:07.510294 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:07.510448 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:07.511196 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:07.512352 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:07.513289 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:07.513836 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:07.513956 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:07.516669 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:07.516673 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:07.516674 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:07.516686 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:07.518568 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:07.518990 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:07.518993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:07.518994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:07.519111 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:07.519312 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:07.519313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:07.519313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:07.519621 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:07.519774 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:07.519855 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 15:07:07.519965 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260806 15:07:07.521123 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:07:07.529184 32129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:07.522456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:59879 (local address 127.0.0.1:45827)
0806 15:07:07.522487 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:07.522489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:07.522603 (+ 114us) server_negotiation.cc:407] Connection header received
0806 15:07:07.522650 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:07.522651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:07.522658 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:07.522709 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:07.523288 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:07.524001 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:07.524967 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:07.525263 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:07.525323 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:07.526646 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:07.526647 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:07.526647 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:07.526654 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:07.528059 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:07.528462 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:07.528463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:07.528463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:07.528511 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:07.528715 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:07.528715 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:07.528716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:07.528820 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:07.528928 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:07.529044 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 15:07:07.529100 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 15:07:07.550194 32130 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:07:07.550671 32130 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:07:07.553820 32130 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1bfbebae1aed4b0f8cc021d8b0af0f92: Generated new TSK 0
I20260806 15:07:07.554013 32130 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:07:07.560882 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 32011
W20260806 15:07:07.574714 31990 connection.cc:535] client connection to 127.0.0.1:45827 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45827): Connection reset by peer (error 104)
2026-08-06T15:07:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1391 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:07:07 dist-test-slave-tc2s krb5kdc[32156](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:07:07 dist-test-slave-tc2s krb5kdc[32156](info): set up 2 sockets
Aug 06 15:07:07 dist-test-slave-tc2s krb5kdc[32156](info): commencing operation
krb5kdc: starting...
W20260806 15:07:08.690881 945 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:07:08 dist-test-slave-tc2s krb5kdc[32156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:07:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:07:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 15:07:08.802139 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45153
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:07:08.880254 32177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:07:08.880451 32177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:07:08.880472 32177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:07:08.881561 32177 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:07:08.881600 32177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:07:08.881613 32177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:07:08.881630 32177 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:07:08.881654 32177 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:07:08.883105 32177 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:07:08.883315 32177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:07:08.883497 32177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:07:08.887899 32185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:08.887940 32177 server_base.cc:1034] running on GCE node
W20260806 15:07:08.887917 32187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:08.887907 32183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:08.888330 32177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:07:08.888610 32177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:07:08.889745 32177 hybrid_clock.cc:648] HybridClock initialized: now 1786028828889732 us; error 34 us; skew 500 ppm
Aug 06 15:07:08 dist-test-slave-tc2s krb5kdc[32156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028828, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:07:08.892014 32177 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 15:07:08.892402 32177 webserver.cc:460] Webserver started at http://127.0.0.1:34183/ using document root <none> and password file <none>
I20260806 15:07:08.892601 32177 fs_manager.cc:359] Metadata directory not provided
I20260806 15:07:08.892642 32177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:07:08.892752 32177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:07:08.893656 32177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "c39f5fba9aa7494baa60e97d6bc346e9"
format_stamp: "Formatted at 2026-08-06 15:07:08 on dist-test-slave-tc2s"
server_key: "63242fd72340bbfcb6fa3fe8a4204472"
server_key_iv: "aa666e20a2c96b1b260a10b6b50271cc"
server_key_version: "encryptionkey@0"
I20260806 15:07:08.893989 32177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "c39f5fba9aa7494baa60e97d6bc346e9"
format_stamp: "Formatted at 2026-08-06 15:07:08 on dist-test-slave-tc2s"
server_key: "63242fd72340bbfcb6fa3fe8a4204472"
server_key_iv: "aa666e20a2c96b1b260a10b6b50271cc"
server_key_version: "encryptionkey@0"
I20260806 15:07:08.895136 32177 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:07:08.895767 32197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:08.895908 32177 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:08.895980 32177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "c39f5fba9aa7494baa60e97d6bc346e9"
format_stamp: "Formatted at 2026-08-06 15:07:08 on dist-test-slave-tc2s"
server_key: "63242fd72340bbfcb6fa3fe8a4204472"
server_key_iv: "aa666e20a2c96b1b260a10b6b50271cc"
server_key_version: "encryptionkey@0"
I20260806 15:07:08.896049 32177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:07:08.906841 32177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:07:08.908715 32177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:07:08.908833 32177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:07:08.912494 32177 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45153
I20260806 15:07:08.912572 32277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45153 every 8 connection(s)
I20260806 15:07:08.912830 32177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:07:08.913581 32278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:07:08.915644 32278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c39f5fba9aa7494baa60e97d6bc346e9: Bootstrap starting.
I20260806 15:07:08.916095 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 32177
I20260806 15:07:08.916184 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:07:08.916208 32278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c39f5fba9aa7494baa60e97d6bc346e9: Neither blocks nor log segments found. Creating new log.
I20260806 15:07:08.916360 945 external_mini_cluster.cc:1428] Setting key 490e05fd096a91d69cd015c28e0a6e58
I20260806 15:07:08.916494 32278 log.cc:824] T 00000000000000000000000000000000 P c39f5fba9aa7494baa60e97d6bc346e9: Log is configured to *not* fsync() on all Append() calls
I20260806 15:07:08.917196 32278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c39f5fba9aa7494baa60e97d6bc346e9: No bootstrap required, opened a new log
I20260806 15:07:08.919209 32278 raft_consensus.cc:348] T 00000000000000000000000000000000 P c39f5fba9aa7494baa60e97d6bc346e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c39f5fba9aa7494baa60e97d6bc346e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45153 } }
I20260806 15:07:08.919366 32278 raft_consensus.cc:374] T 00000000000000000000000000000000 P c39f5fba9aa7494baa60e97d6bc346e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:07:08.919394 32278 raft_consensus.cc:729] T 00000000000000000000000000000000 P c39f5fba9aa7494baa60e97d6bc346e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c39f5fba9aa7494baa60e97d6bc346e9, State: Initialized, Role: FOLLOWER
I20260806 15:07:08.919514 32278 consensus_queue.cc:260] T 00000000000000000000000000000000 P c39f5fba9aa7494baa60e97d6bc346e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c39f5fba9aa7494baa60e97d6bc346e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45153 } }
I20260806 15:07:08.919582 32278 raft_consensus.cc:388] T 00000000000000000000000000000000 P c39f5fba9aa7494baa60e97d6bc346e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:07:08.919616 32278 raft_consensus.cc:482] T 00000000000000000000000000000000 P c39f5fba9aa7494baa60e97d6bc346e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:07:08.919657 32278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c39f5fba9aa7494baa60e97d6bc346e9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:07:08.920267 32278 raft_consensus.cc:504] T 00000000000000000000000000000000 P c39f5fba9aa7494baa60e97d6bc346e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c39f5fba9aa7494baa60e97d6bc346e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45153 } }
I20260806 15:07:08.920388 32278 leader_election.cc:302] T 00000000000000000000000000000000 P c39f5fba9aa7494baa60e97d6bc346e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c39f5fba9aa7494baa60e97d6bc346e9; no voters:
I20260806 15:07:08.920552 32278 leader_election.cc:288] T 00000000000000000000000000000000 P c39f5fba9aa7494baa60e97d6bc346e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:07:08.920589 32284 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c39f5fba9aa7494baa60e97d6bc346e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:07:08.920764 32284 raft_consensus.cc:686] T 00000000000000000000000000000000 P c39f5fba9aa7494baa60e97d6bc346e9 [term 1 LEADER]: Becoming Leader. State: Replica: c39f5fba9aa7494baa60e97d6bc346e9, State: Running, Role: LEADER
I20260806 15:07:08.920833 32278 sys_catalog.cc:564] T 00000000000000000000000000000000 P c39f5fba9aa7494baa60e97d6bc346e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:07:08.920930 32284 consensus_queue.cc:237] T 00000000000000000000000000000000 P c39f5fba9aa7494baa60e97d6bc346e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c39f5fba9aa7494baa60e97d6bc346e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45153 } }
I20260806 15:07:08.921260 32284 sys_catalog.cc:455] T 00000000000000000000000000000000 P c39f5fba9aa7494baa60e97d6bc346e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c39f5fba9aa7494baa60e97d6bc346e9. Latest consensus state: current_term: 1 leader_uuid: "c39f5fba9aa7494baa60e97d6bc346e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c39f5fba9aa7494baa60e97d6bc346e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45153 } } }
I20260806 15:07:08.921358 32284 sys_catalog.cc:458] T 00000000000000000000000000000000 P c39f5fba9aa7494baa60e97d6bc346e9 [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:08.921267 32286 sys_catalog.cc:455] T 00000000000000000000000000000000 P c39f5fba9aa7494baa60e97d6bc346e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c39f5fba9aa7494baa60e97d6bc346e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c39f5fba9aa7494baa60e97d6bc346e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45153 } } }
I20260806 15:07:08.921471 32286 sys_catalog.cc:458] T 00000000000000000000000000000000 P c39f5fba9aa7494baa60e97d6bc346e9 [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:08.921564 32301 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:07:08.922021 32283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:08.916774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:37221 (local address 127.0.0.1:45153)
0806 15:07:08.916923 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:08.916932 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:08.916965 (+ 33us) server_negotiation.cc:407] Connection header received
0806 15:07:08.917405 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:08.917411 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:08.917457 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:08.917566 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:08.917788 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:08.918675 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:08.919762 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:08.920387 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:08.920502 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:08.921270 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:08.921275 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:08.921275 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:07:08.921476 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:08.921515 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:07:08.921531 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:08.921621 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:08.921762 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 15:07:08.921792 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":69,"threads_started":1}
I20260806 15:07:08.922677 32301 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:07:08.924080 32301 catalog_manager.cc:1347] Generated new cluster ID: 2d09432503dd4c7d9ac84aa65d255d17
I20260806 15:07:08.924139 32301 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:07:08.926704 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:07:08.930742 32283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:08.927905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:41391 (local address 127.0.0.1:45153)
0806 15:07:08.927939 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:08.927940 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:08.928056 (+ 116us) server_negotiation.cc:407] Connection header received
0806 15:07:08.928113 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:08.928114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:08.928123 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:08.928166 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:08.928283 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:08.928999 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:08.929658 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:08.929882 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:08.929954 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:08.930530 (+ 576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:08.930531 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:08.930532 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:07:08.930536 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:08.930548 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:07:08.930552 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:08.930601 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:08.930667 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 15:07:08.930675 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:07:08.933884 32301 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:07:08.934588 32301 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:07:08.946352 32301 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c39f5fba9aa7494baa60e97d6bc346e9: Generated new TSK 0
I20260806 15:07:08.946506 32301 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:07:08.948856 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 32177
W20260806 15:07:08.971204 32154 connection.cc:535] client connection to 127.0.0.1:45153 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45153): Connection reset by peer (error 104)
2026-08-06T15:07:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1396 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:07:09 dist-test-slave-tc2s krb5kdc[32324](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:07:09 dist-test-slave-tc2s krb5kdc[32324](info): set up 2 sockets
Aug 06 15:07:09 dist-test-slave-tc2s krb5kdc[32324](info): commencing operation
krb5kdc: starting...
W20260806 15:07:10.067276 945 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:07:10 dist-test-slave-tc2s krb5kdc[32324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:07:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:07:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 15:07:10.174705 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38075
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:07:10.253439 32347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:07:10.253623 32347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:07:10.253643 32347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:07:10.254711 32347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:07:10.254746 32347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:07:10.254760 32347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:07:10.254770 32347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:07:10.254786 32347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:07:10.256177 32347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:07:10.256354 32347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:07:10.256517 32347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:07:10.260931 32355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:10.260939 32358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:10.260977 32356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:10.261214 32347 server_base.cc:1034] running on GCE node
I20260806 15:07:10.261433 32347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:07:10.261706 32347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:07:10.262840 32347 hybrid_clock.cc:648] HybridClock initialized: now 1786028830262815 us; error 42 us; skew 500 ppm
Aug 06 15:07:10 dist-test-slave-tc2s krb5kdc[32324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028830, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:07:10.271549 32347 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 15:07:10.271946 32347 webserver.cc:460] Webserver started at http://127.0.0.1:37015/ using document root <none> and password file <none>
I20260806 15:07:10.272156 32347 fs_manager.cc:359] Metadata directory not provided
I20260806 15:07:10.272204 32347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:07:10.272328 32347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:07:10.273316 32347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "a413f112faf049fcba105ae78615d1fc"
format_stamp: "Formatted at 2026-08-06 15:07:10 on dist-test-slave-tc2s"
server_key: "6f6823962bd9b8213cafb6cd8c0047c9"
server_key_iv: "a9a7cfc11f24958fe85225fa430d0913"
server_key_version: "encryptionkey@0"
I20260806 15:07:10.273624 32347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "a413f112faf049fcba105ae78615d1fc"
format_stamp: "Formatted at 2026-08-06 15:07:10 on dist-test-slave-tc2s"
server_key: "6f6823962bd9b8213cafb6cd8c0047c9"
server_key_iv: "a9a7cfc11f24958fe85225fa430d0913"
server_key_version: "encryptionkey@0"
I20260806 15:07:10.274891 32347 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:07:10.275554 32365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:10.275691 32347 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:07:10.275774 32347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "a413f112faf049fcba105ae78615d1fc"
format_stamp: "Formatted at 2026-08-06 15:07:10 on dist-test-slave-tc2s"
server_key: "6f6823962bd9b8213cafb6cd8c0047c9"
server_key_iv: "a9a7cfc11f24958fe85225fa430d0913"
server_key_version: "encryptionkey@0"
I20260806 15:07:10.275839 32347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:07:10.279036 32347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:07:10.280674 32347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:07:10.280797 32347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:07:10.284452 32347 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38075
I20260806 15:07:10.284482 32439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38075 every 8 connection(s)
I20260806 15:07:10.284758 32347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:07:10.285684 32440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:07:10.287734 32440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a413f112faf049fcba105ae78615d1fc: Bootstrap starting.
I20260806 15:07:10.288236 32440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a413f112faf049fcba105ae78615d1fc: Neither blocks nor log segments found. Creating new log.
I20260806 15:07:10.288463 32440 log.cc:824] T 00000000000000000000000000000000 P a413f112faf049fcba105ae78615d1fc: Log is configured to *not* fsync() on all Append() calls
I20260806 15:07:10.288635 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 32347
I20260806 15:07:10.288727 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:07:10.288856 945 external_mini_cluster.cc:1428] Setting key 454209bc01f3920b16859ce7a62a6de3
I20260806 15:07:10.289094 32440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a413f112faf049fcba105ae78615d1fc: No bootstrap required, opened a new log
I20260806 15:07:10.291112 32440 raft_consensus.cc:348] T 00000000000000000000000000000000 P a413f112faf049fcba105ae78615d1fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a413f112faf049fcba105ae78615d1fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38075 } }
I20260806 15:07:10.291273 32440 raft_consensus.cc:374] T 00000000000000000000000000000000 P a413f112faf049fcba105ae78615d1fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:07:10.291327 32440 raft_consensus.cc:729] T 00000000000000000000000000000000 P a413f112faf049fcba105ae78615d1fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a413f112faf049fcba105ae78615d1fc, State: Initialized, Role: FOLLOWER
I20260806 15:07:10.291471 32440 consensus_queue.cc:260] T 00000000000000000000000000000000 P a413f112faf049fcba105ae78615d1fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a413f112faf049fcba105ae78615d1fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38075 } }
I20260806 15:07:10.291568 32440 raft_consensus.cc:388] T 00000000000000000000000000000000 P a413f112faf049fcba105ae78615d1fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:07:10.291610 32440 raft_consensus.cc:482] T 00000000000000000000000000000000 P a413f112faf049fcba105ae78615d1fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:07:10.291617 32445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:10.289265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:50151 (local address 127.0.0.1:38075)
0806 15:07:10.289423 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:10.289432 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:10.289468 (+ 36us) server_negotiation.cc:407] Connection header received
0806 15:07:10.290299 (+ 831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:10.290308 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:10.290341 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:10.290485 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:10.290992 (+ 507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:10.290995 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:10.290996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:07:10.291000 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:10.291041 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:07:10.291055 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:10.291127 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:10.291167 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 15:07:10.291206 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":75,"threads_started":1}
I20260806 15:07:10.291661 32440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a413f112faf049fcba105ae78615d1fc [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:07:10.292208 32440 raft_consensus.cc:504] T 00000000000000000000000000000000 P a413f112faf049fcba105ae78615d1fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a413f112faf049fcba105ae78615d1fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38075 } }
I20260806 15:07:10.292312 32440 leader_election.cc:302] T 00000000000000000000000000000000 P a413f112faf049fcba105ae78615d1fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a413f112faf049fcba105ae78615d1fc; no voters:
I20260806 15:07:10.292475 32440 leader_election.cc:288] T 00000000000000000000000000000000 P a413f112faf049fcba105ae78615d1fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:07:10.292476 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:07:10.292622 32447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a413f112faf049fcba105ae78615d1fc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:07:10.292703 32440 sys_catalog.cc:564] T 00000000000000000000000000000000 P a413f112faf049fcba105ae78615d1fc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:07:10.292743 32447 raft_consensus.cc:686] T 00000000000000000000000000000000 P a413f112faf049fcba105ae78615d1fc [term 1 LEADER]: Becoming Leader. State: Replica: a413f112faf049fcba105ae78615d1fc, State: Running, Role: LEADER
I20260806 15:07:10.292883 32447 consensus_queue.cc:237] T 00000000000000000000000000000000 P a413f112faf049fcba105ae78615d1fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a413f112faf049fcba105ae78615d1fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38075 } }
I20260806 15:07:10.293236 32447 sys_catalog.cc:455] T 00000000000000000000000000000000 P a413f112faf049fcba105ae78615d1fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader a413f112faf049fcba105ae78615d1fc. Latest consensus state: current_term: 1 leader_uuid: "a413f112faf049fcba105ae78615d1fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a413f112faf049fcba105ae78615d1fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38075 } } }
I20260806 15:07:10.293457 32447 sys_catalog.cc:458] T 00000000000000000000000000000000 P a413f112faf049fcba105ae78615d1fc [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:10.293385 32448 sys_catalog.cc:455] T 00000000000000000000000000000000 P a413f112faf049fcba105ae78615d1fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a413f112faf049fcba105ae78615d1fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a413f112faf049fcba105ae78615d1fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38075 } } }
I20260806 15:07:10.293694 32448 sys_catalog.cc:458] T 00000000000000000000000000000000 P a413f112faf049fcba105ae78615d1fc [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:10.294003 32463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:07:10.294479 32463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:07:10.295224 32445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:10.294213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:55039 (local address 127.0.0.1:38075)
0806 15:07:10.294250 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:10.294252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:10.294360 (+ 108us) server_negotiation.cc:407] Connection header received
0806 15:07:10.294462 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:10.294463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:10.294468 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:10.294496 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:10.295045 (+ 549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:10.295046 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:10.295047 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:07:10.295050 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:10.295061 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:07:10.295065 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:10.295110 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:10.295158 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 15:07:10.295168 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:07:10.296099 32463 catalog_manager.cc:1347] Generated new cluster ID: 9e573dbbf4a94edfbbe57f0baa6b7a18
I20260806 15:07:10.296140 32463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:07:10.304720 32463 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:07:10.304849 32463 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:07:10.313989 32463 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a413f112faf049fcba105ae78615d1fc: Generated new TSK 0
I20260806 15:07:10.314127 32463 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:07:10.320477 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 32347
2026-08-06T15:07:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1363 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:07:10 dist-test-slave-tc2s krb5kdc[32477](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:07:10 dist-test-slave-tc2s krb5kdc[32477](info): set up 2 sockets
Aug 06 15:07:10 dist-test-slave-tc2s krb5kdc[32477](info): commencing operation
krb5kdc: starting...
W20260806 15:07:11.430660 945 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:07:11 dist-test-slave-tc2s krb5kdc[32477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:07:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:07:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 15:07:11.536206 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40763
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40763
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:07:11.613412 32494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:07:11.613597 32494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:07:11.613628 32494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:07:11.614729 32494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:07:11.614768 32494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:07:11.614789 32494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:07:11.614804 32494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:07:11.614820 32494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:07:11.616233 32494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40763
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:07:11.616462 32494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:07:11.616652 32494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:07:11.621093 32503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:11.621104 32501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:11.621273 32494 server_base.cc:1034] running on GCE node
W20260806 15:07:11.621104 32505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:11.621527 32494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:07:11.621776 32494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:07:11.622906 32494 hybrid_clock.cc:648] HybridClock initialized: now 1786028831622890 us; error 32 us; skew 500 ppm
Aug 06 15:07:11 dist-test-slave-tc2s krb5kdc[32477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028831, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:07:11.625092 32494 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 15:07:11.625496 32494 webserver.cc:460] Webserver started at http://127.0.0.1:42057/ using document root <none> and password file <none>
I20260806 15:07:11.625694 32494 fs_manager.cc:359] Metadata directory not provided
I20260806 15:07:11.625739 32494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:07:11.625859 32494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:07:11.626812 32494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "c809ce9e2f614b20a6b527685274e1a3"
format_stamp: "Formatted at 2026-08-06 15:07:11 on dist-test-slave-tc2s"
server_key: "758de58a90c29676521a4034e567930b"
server_key_iv: "79d5e4e27406a0a8f4d7c7363b239317"
server_key_version: "encryptionkey@0"
I20260806 15:07:11.627130 32494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "c809ce9e2f614b20a6b527685274e1a3"
format_stamp: "Formatted at 2026-08-06 15:07:11 on dist-test-slave-tc2s"
server_key: "758de58a90c29676521a4034e567930b"
server_key_iv: "79d5e4e27406a0a8f4d7c7363b239317"
server_key_version: "encryptionkey@0"
I20260806 15:07:11.628289 32494 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:07:11.628887 32516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:11.629030 32494 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:07:11.629109 32494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "c809ce9e2f614b20a6b527685274e1a3"
format_stamp: "Formatted at 2026-08-06 15:07:11 on dist-test-slave-tc2s"
server_key: "758de58a90c29676521a4034e567930b"
server_key_iv: "79d5e4e27406a0a8f4d7c7363b239317"
server_key_version: "encryptionkey@0"
I20260806 15:07:11.629169 32494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:07:11.650014 32494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:07:11.651546 32494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:07:11.651672 32494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:07:11.655637 32494 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40763
I20260806 15:07:11.655670 32598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40763 every 8 connection(s)
I20260806 15:07:11.655972 32494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:07:11.656754 32599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:07:11.658888 32599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c809ce9e2f614b20a6b527685274e1a3: Bootstrap starting.
I20260806 15:07:11.659417 32599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c809ce9e2f614b20a6b527685274e1a3: Neither blocks nor log segments found. Creating new log.
I20260806 15:07:11.659687 32599 log.cc:824] T 00000000000000000000000000000000 P c809ce9e2f614b20a6b527685274e1a3: Log is configured to *not* fsync() on all Append() calls
I20260806 15:07:11.660053 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 32494
I20260806 15:07:11.660175 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:07:11.660348 945 external_mini_cluster.cc:1428] Setting key 5fa7cfa0bae8bc5c78306a1ecf4db921
I20260806 15:07:11.660383 32599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c809ce9e2f614b20a6b527685274e1a3: No bootstrap required, opened a new log
I20260806 15:07:11.662454 32599 raft_consensus.cc:348] T 00000000000000000000000000000000 P c809ce9e2f614b20a6b527685274e1a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c809ce9e2f614b20a6b527685274e1a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40763 } }
I20260806 15:07:11.662621 32599 raft_consensus.cc:374] T 00000000000000000000000000000000 P c809ce9e2f614b20a6b527685274e1a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:07:11.662669 32599 raft_consensus.cc:729] T 00000000000000000000000000000000 P c809ce9e2f614b20a6b527685274e1a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c809ce9e2f614b20a6b527685274e1a3, State: Initialized, Role: FOLLOWER
I20260806 15:07:11.662798 32599 consensus_queue.cc:260] T 00000000000000000000000000000000 P c809ce9e2f614b20a6b527685274e1a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c809ce9e2f614b20a6b527685274e1a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40763 } }
I20260806 15:07:11.662871 32599 raft_consensus.cc:388] T 00000000000000000000000000000000 P c809ce9e2f614b20a6b527685274e1a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:07:11.662906 32599 raft_consensus.cc:482] T 00000000000000000000000000000000 P c809ce9e2f614b20a6b527685274e1a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:07:11.662949 32599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c809ce9e2f614b20a6b527685274e1a3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:07:11.663595 32599 raft_consensus.cc:504] T 00000000000000000000000000000000 P c809ce9e2f614b20a6b527685274e1a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c809ce9e2f614b20a6b527685274e1a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40763 } }
I20260806 15:07:11.663699 32599 leader_election.cc:302] T 00000000000000000000000000000000 P c809ce9e2f614b20a6b527685274e1a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c809ce9e2f614b20a6b527685274e1a3; no voters:
I20260806 15:07:11.663836 32599 leader_election.cc:288] T 00000000000000000000000000000000 P c809ce9e2f614b20a6b527685274e1a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:07:11.663868 32605 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c809ce9e2f614b20a6b527685274e1a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:07:11.664003 32605 raft_consensus.cc:686] T 00000000000000000000000000000000 P c809ce9e2f614b20a6b527685274e1a3 [term 1 LEADER]: Becoming Leader. State: Replica: c809ce9e2f614b20a6b527685274e1a3, State: Running, Role: LEADER
I20260806 15:07:11.664042 32599 sys_catalog.cc:564] T 00000000000000000000000000000000 P c809ce9e2f614b20a6b527685274e1a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:07:11.665212 32605 consensus_queue.cc:237] T 00000000000000000000000000000000 P c809ce9e2f614b20a6b527685274e1a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c809ce9e2f614b20a6b527685274e1a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40763 } }
I20260806 15:07:11.665853 32609 sys_catalog.cc:455] T 00000000000000000000000000000000 P c809ce9e2f614b20a6b527685274e1a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c809ce9e2f614b20a6b527685274e1a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c809ce9e2f614b20a6b527685274e1a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40763 } } }
I20260806 15:07:11.665928 32609 sys_catalog.cc:458] T 00000000000000000000000000000000 P c809ce9e2f614b20a6b527685274e1a3 [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:11.665736 32608 sys_catalog.cc:455] T 00000000000000000000000000000000 P c809ce9e2f614b20a6b527685274e1a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c809ce9e2f614b20a6b527685274e1a3. Latest consensus state: current_term: 1 leader_uuid: "c809ce9e2f614b20a6b527685274e1a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c809ce9e2f614b20a6b527685274e1a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40763 } } }
I20260806 15:07:11.666112 32608 sys_catalog.cc:458] T 00000000000000000000000000000000 P c809ce9e2f614b20a6b527685274e1a3 [sys.catalog]: This master's current role is: LEADER
W20260806 15:07:11.666177 32628 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c809ce9e2f614b20a6b527685274e1a3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:07:11.666246 32628 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:07:11.666309 32631 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:07:11.666483 32631 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 15:07:11 dist-test-slave-tc2s krb5kdc[32477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 15:07:11.667881 32631 catalog_manager.cc:1347] Generated new cluster ID: e7544a42b6fe4458bf14f64cca9cc9a7
I20260806 15:07:11.667927 32631 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:07:11.672158 32604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:11.660793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:51207 (local address 127.0.0.1:40763)
0806 15:07:11.660986 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:11.660995 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:11.661031 (+ 36us) server_negotiation.cc:407] Connection header received
0806 15:07:11.661672 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:11.661682 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:11.661744 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:11.661884 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:11.662444 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:11.664063 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:11.664952 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:11.665446 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:11.665541 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:11.668104 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:11.668107 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:11.668108 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:11.668118 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:11.670065 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:11.670525 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:11.670528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:11.670528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:11.670634 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:11.670880 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:11.670881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:11.670881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:11.671186 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:11.671340 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:11.671837 (+ 497us) server_negotiation.cc:299] Negotiation successful
0806 15:07:11.671950 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":87,"threads_started":1}
I20260806 15:07:11.673547 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:07:11.681579 32604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:11.674918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:51599 (local address 127.0.0.1:40763)
0806 15:07:11.674953 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:11.674955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:11.675157 (+ 202us) server_negotiation.cc:407] Connection header received
0806 15:07:11.675189 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:11.675190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:11.675200 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:11.675263 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:11.675791 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:11.676531 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:11.677466 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:11.677778 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:11.677840 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:11.679075 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:11.679076 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:11.679076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:11.679083 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:11.680502 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:11.680872 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:11.680873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:11.680873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:11.680928 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:11.681118 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:11.681118 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:11.681119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:11.681239 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:11.681334 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:11.681432 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 15:07:11.681493 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 15:07:11.685141 32631 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:07:11.685591 32631 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:07:11.690336 32631 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c809ce9e2f614b20a6b527685274e1a3: Generated new TSK 0
I20260806 15:07:11.690488 32631 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:07:11.692165 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 32494
W20260806 15:07:11.716341 32475 connection.cc:535] client connection to 127.0.0.1:40763 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40763): Connection reset by peer (error 104)
2026-08-06T15:07:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1381 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:07:11 dist-test-slave-tc2s krb5kdc[32650](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:07:11 dist-test-slave-tc2s krb5kdc[32650](info): set up 2 sockets
Aug 06 15:07:11 dist-test-slave-tc2s krb5kdc[32650](info): commencing operation
krb5kdc: starting...
W20260806 15:07:12.813724 945 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:07:12 dist-test-slave-tc2s krb5kdc[32650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:07:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:07:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 15:07:12.922196 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37503
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:07:13.000500 32668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:07:13.000679 32668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:07:13.000700 32668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:07:13.001784 32668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:07:13.001832 32668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:07:13.001847 32668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:07:13.001871 32668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:07:13.001916 32668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:07:13.003381 32668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37503
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:07:13.003594 32668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:07:13.003791 32668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:07:13.008208 32677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:13.008225 32675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:13.008225 32683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:13.008441 32668 server_base.cc:1034] running on GCE node
I20260806 15:07:13.008678 32668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:07:13.008949 32668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:07:13.010080 32668 hybrid_clock.cc:648] HybridClock initialized: now 1786028833010068 us; error 30 us; skew 500 ppm
Aug 06 15:07:13 dist-test-slave-tc2s krb5kdc[32650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028833, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:07:13.012390 32668 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 15:07:13.012748 32668 webserver.cc:460] Webserver started at http://127.0.0.1:33287/ using document root <none> and password file <none>
I20260806 15:07:13.012945 32668 fs_manager.cc:359] Metadata directory not provided
I20260806 15:07:13.012985 32668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:07:13.013073 32668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:07:13.014037 32668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "5be39a122f8f480dae59f6e49fd96b15"
format_stamp: "Formatted at 2026-08-06 15:07:13 on dist-test-slave-tc2s"
server_key: "5d3ca03984dfbbdef743999a3d41b4b1"
server_key_iv: "2e339e5a74eed081d826aecac4734e64"
server_key_version: "encryptionkey@0"
I20260806 15:07:13.014307 32668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "5be39a122f8f480dae59f6e49fd96b15"
format_stamp: "Formatted at 2026-08-06 15:07:13 on dist-test-slave-tc2s"
server_key: "5d3ca03984dfbbdef743999a3d41b4b1"
server_key_iv: "2e339e5a74eed081d826aecac4734e64"
server_key_version: "encryptionkey@0"
I20260806 15:07:13.015419 32668 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:07:13.016047 32695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:13.016211 32668 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:07:13.016287 32668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "5be39a122f8f480dae59f6e49fd96b15"
format_stamp: "Formatted at 2026-08-06 15:07:13 on dist-test-slave-tc2s"
server_key: "5d3ca03984dfbbdef743999a3d41b4b1"
server_key_iv: "2e339e5a74eed081d826aecac4734e64"
server_key_version: "encryptionkey@0"
I20260806 15:07:13.016340 32668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:07:13.026829 32668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:07:13.028359 32668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:07:13.028460 32668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:07:13.032173 32668 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37503
I20260806 15:07:13.032208 311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37503 every 8 connection(s)
I20260806 15:07:13.032492 32668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:07:13.033404 314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:07:13.035526 314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5be39a122f8f480dae59f6e49fd96b15: Bootstrap starting.
I20260806 15:07:13.036059 314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5be39a122f8f480dae59f6e49fd96b15: Neither blocks nor log segments found. Creating new log.
I20260806 15:07:13.036177 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 32668
I20260806 15:07:13.036291 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:07:13.036336 314 log.cc:824] T 00000000000000000000000000000000 P 5be39a122f8f480dae59f6e49fd96b15: Log is configured to *not* fsync() on all Append() calls
I20260806 15:07:13.036482 945 external_mini_cluster.cc:1428] Setting key 77168a13aef591f4dd69b3b0176b9e9b
I20260806 15:07:13.037083 314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5be39a122f8f480dae59f6e49fd96b15: No bootstrap required, opened a new log
I20260806 15:07:13.039664 314 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5be39a122f8f480dae59f6e49fd96b15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5be39a122f8f480dae59f6e49fd96b15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37503 } }
I20260806 15:07:13.039786 314 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5be39a122f8f480dae59f6e49fd96b15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:07:13.039806 314 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5be39a122f8f480dae59f6e49fd96b15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5be39a122f8f480dae59f6e49fd96b15, State: Initialized, Role: FOLLOWER
I20260806 15:07:13.039909 314 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5be39a122f8f480dae59f6e49fd96b15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5be39a122f8f480dae59f6e49fd96b15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37503 } }
I20260806 15:07:13.039974 314 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5be39a122f8f480dae59f6e49fd96b15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:07:13.039997 314 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5be39a122f8f480dae59f6e49fd96b15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:07:13.040040 314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5be39a122f8f480dae59f6e49fd96b15 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:07:13.040680 314 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5be39a122f8f480dae59f6e49fd96b15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5be39a122f8f480dae59f6e49fd96b15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37503 } }
I20260806 15:07:13.040784 314 leader_election.cc:302] T 00000000000000000000000000000000 P 5be39a122f8f480dae59f6e49fd96b15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5be39a122f8f480dae59f6e49fd96b15; no voters:
I20260806 15:07:13.040935 314 leader_election.cc:288] T 00000000000000000000000000000000 P 5be39a122f8f480dae59f6e49fd96b15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:07:13.041096 321 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5be39a122f8f480dae59f6e49fd96b15 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:07:13.041285 321 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5be39a122f8f480dae59f6e49fd96b15 [term 1 LEADER]: Becoming Leader. State: Replica: 5be39a122f8f480dae59f6e49fd96b15, State: Running, Role: LEADER
I20260806 15:07:13.041414 321 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5be39a122f8f480dae59f6e49fd96b15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5be39a122f8f480dae59f6e49fd96b15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37503 } }
I20260806 15:07:13.041452 314 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5be39a122f8f480dae59f6e49fd96b15 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:07:13.041756 322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5be39a122f8f480dae59f6e49fd96b15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5be39a122f8f480dae59f6e49fd96b15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5be39a122f8f480dae59f6e49fd96b15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37503 } } }
I20260806 15:07:13.041832 322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5be39a122f8f480dae59f6e49fd96b15 [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:13.042083 323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5be39a122f8f480dae59f6e49fd96b15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5be39a122f8f480dae59f6e49fd96b15. Latest consensus state: current_term: 1 leader_uuid: "5be39a122f8f480dae59f6e49fd96b15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5be39a122f8f480dae59f6e49fd96b15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37503 } } }
I20260806 15:07:13.042201 323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5be39a122f8f480dae59f6e49fd96b15 [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:13.042415 318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:13.036905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:44789 (local address 127.0.0.1:37503)
0806 15:07:13.037083 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:13.037092 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:13.037124 (+ 32us) server_negotiation.cc:407] Connection header received
0806 15:07:13.037587 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:13.037596 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:13.037656 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:13.037797 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:13.037964 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:13.039347 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:13.040216 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:13.040834 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:13.040959 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:13.041614 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:13.041618 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:13.041618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:07:13.041813 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:13.041852 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:07:13.041867 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:13.041986 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:13.042206 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 15:07:13.042236 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
I20260806 15:07:13.042641 333 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:07:13.043198 333 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:07:13.043349 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:07:13.044521 333 catalog_manager.cc:1347] Generated new cluster ID: 2b533651a57d438a900ca85e0335ad91
I20260806 15:07:13.044579 333 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:07:13.053249 318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:13.050095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:37629 (local address 127.0.0.1:37503)
0806 15:07:13.050129 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:13.050130 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:13.050244 (+ 114us) server_negotiation.cc:407] Connection header received
0806 15:07:13.050311 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:13.050311 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:13.050322 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:13.050364 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:13.050494 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:13.051066 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:13.052016 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:13.052350 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:13.052426 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:13.053024 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:13.053026 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:13.053026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:07:13.053031 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:13.053045 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:07:13.053049 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:13.053110 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:13.053167 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 15:07:13.053178 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 15:07:13.071493 333 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:07:13.072173 333 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:07:13.094074 333 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5be39a122f8f480dae59f6e49fd96b15: Generated new TSK 0
I20260806 15:07:13.094269 333 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:07:13.104465 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 32668
W20260806 15:07:13.116703 32645 connection.cc:535] client connection to 127.0.0.1:37503 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37503): Connection reset by peer (error 104)
2026-08-06T15:07:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1400 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:07:13 dist-test-slave-tc2s krb5kdc[362](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:07:13 dist-test-slave-tc2s krb5kdc[362](info): set up 2 sockets
Aug 06 15:07:13 dist-test-slave-tc2s krb5kdc[362](info): commencing operation
krb5kdc: starting...
W20260806 15:07:14.220038 945 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:07:14 dist-test-slave-tc2s krb5kdc[362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:07:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:07:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 15:07:14.328982 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39507
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:07:14.407668 378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:07:14.407857 378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:07:14.407879 378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:07:14.408952 378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:07:14.408986 378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:07:14.408999 378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:07:14.409010 378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:07:14.409025 378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:07:14.410465 378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39507
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:07:14.410657 378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:07:14.410820 378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:07:14.415354 385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:14.415462 378 server_base.cc:1034] running on GCE node
W20260806 15:07:14.415326 387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:14.415534 391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:14.415822 378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:07:14.416090 378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:07:14.417228 378 hybrid_clock.cc:648] HybridClock initialized: now 1786028834417215 us; error 28 us; skew 500 ppm
Aug 06 15:07:14 dist-test-slave-tc2s krb5kdc[362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028834, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:07:14.419579 378 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 15:07:14.419976 378 webserver.cc:460] Webserver started at http://127.0.0.1:46659/ using document root <none> and password file <none>
I20260806 15:07:14.420171 378 fs_manager.cc:359] Metadata directory not provided
I20260806 15:07:14.420217 378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:07:14.420351 378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:07:14.421334 378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "5ace6784563a4c8c815966f8528d814e"
format_stamp: "Formatted at 2026-08-06 15:07:14 on dist-test-slave-tc2s"
server_key: "bf9e1f06a78b8b6a4a3e174fcb330d6f"
server_key_iv: "4217d8b8fe5137a7451a29ba1f0dc07b"
server_key_version: "encryptionkey@0"
I20260806 15:07:14.421661 378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "5ace6784563a4c8c815966f8528d814e"
format_stamp: "Formatted at 2026-08-06 15:07:14 on dist-test-slave-tc2s"
server_key: "bf9e1f06a78b8b6a4a3e174fcb330d6f"
server_key_iv: "4217d8b8fe5137a7451a29ba1f0dc07b"
server_key_version: "encryptionkey@0"
I20260806 15:07:14.422987 378 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:07:14.423650 403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:14.423795 378 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:07:14.423879 378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "5ace6784563a4c8c815966f8528d814e"
format_stamp: "Formatted at 2026-08-06 15:07:14 on dist-test-slave-tc2s"
server_key: "bf9e1f06a78b8b6a4a3e174fcb330d6f"
server_key_iv: "4217d8b8fe5137a7451a29ba1f0dc07b"
server_key_version: "encryptionkey@0"
I20260806 15:07:14.423944 378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:07:14.427227 378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:07:14.428815 378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:07:14.428936 378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:07:14.432674 378 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39507
I20260806 15:07:14.432703 482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39507 every 8 connection(s)
I20260806 15:07:14.432976 378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:07:14.433866 483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:07:14.435969 483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ace6784563a4c8c815966f8528d814e: Bootstrap starting.
I20260806 15:07:14.436605 483 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ace6784563a4c8c815966f8528d814e: Neither blocks nor log segments found. Creating new log.
I20260806 15:07:14.436854 483 log.cc:824] T 00000000000000000000000000000000 P 5ace6784563a4c8c815966f8528d814e: Log is configured to *not* fsync() on all Append() calls
I20260806 15:07:14.437438 483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ace6784563a4c8c815966f8528d814e: No bootstrap required, opened a new log
I20260806 15:07:14.439070 483 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ace6784563a4c8c815966f8528d814e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ace6784563a4c8c815966f8528d814e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39507 } }
I20260806 15:07:14.439198 483 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ace6784563a4c8c815966f8528d814e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:07:14.439219 483 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ace6784563a4c8c815966f8528d814e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ace6784563a4c8c815966f8528d814e, State: Initialized, Role: FOLLOWER
I20260806 15:07:14.439323 483 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ace6784563a4c8c815966f8528d814e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ace6784563a4c8c815966f8528d814e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39507 } }
I20260806 15:07:14.439396 483 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ace6784563a4c8c815966f8528d814e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:07:14.439424 483 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ace6784563a4c8c815966f8528d814e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:07:14.439461 483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ace6784563a4c8c815966f8528d814e [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:07:14.439903 483 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ace6784563a4c8c815966f8528d814e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ace6784563a4c8c815966f8528d814e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39507 } }
I20260806 15:07:14.439985 483 leader_election.cc:302] T 00000000000000000000000000000000 P 5ace6784563a4c8c815966f8528d814e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ace6784563a4c8c815966f8528d814e; no voters:
I20260806 15:07:14.440104 483 leader_election.cc:288] T 00000000000000000000000000000000 P 5ace6784563a4c8c815966f8528d814e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:07:14.440155 487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ace6784563a4c8c815966f8528d814e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:07:14.440313 483 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ace6784563a4c8c815966f8528d814e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:07:14.440531 487 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ace6784563a4c8c815966f8528d814e [term 1 LEADER]: Becoming Leader. State: Replica: 5ace6784563a4c8c815966f8528d814e, State: Running, Role: LEADER
I20260806 15:07:14.440622 487 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ace6784563a4c8c815966f8528d814e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ace6784563a4c8c815966f8528d814e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39507 } }
I20260806 15:07:14.440963 489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ace6784563a4c8c815966f8528d814e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ace6784563a4c8c815966f8528d814e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ace6784563a4c8c815966f8528d814e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39507 } } }
I20260806 15:07:14.441057 489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ace6784563a4c8c815966f8528d814e [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:14.440973 491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ace6784563a4c8c815966f8528d814e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ace6784563a4c8c815966f8528d814e. Latest consensus state: current_term: 1 leader_uuid: "5ace6784563a4c8c815966f8528d814e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ace6784563a4c8c815966f8528d814e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39507 } } }
I20260806 15:07:14.441136 491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ace6784563a4c8c815966f8528d814e [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:14.441301 505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:07:14.441808 505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:07:14.442698 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 378
I20260806 15:07:14.442813 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:07:14.442996 945 external_mini_cluster.cc:1428] Setting key 95b4352c8da1a14060143d65e1192745
I20260806 15:07:14.443583 505 catalog_manager.cc:1347] Generated new cluster ID: 59bfd8458fe74f10a1f830a39bf8da60
I20260806 15:07:14.443672 505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:07:14.447294 520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:14.443416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:40859 (local address 127.0.0.1:39507)
0806 15:07:14.446023 (+ 2607us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:14.446031 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:14.446063 (+ 32us) server_negotiation.cc:407] Connection header received
0806 15:07:14.446165 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:14.446171 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:14.446210 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:14.446331 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:14.446924 (+ 593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:14.446927 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:14.446927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:07:14.446930 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:14.446963 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:07:14.446975 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:14.447034 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:14.447100 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 15:07:14.447137 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2536,"thread_start_us":74,"threads_started":1}
I20260806 15:07:14.448173 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:07:14.450839 505 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:07:14.450966 505 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:07:14.453473 505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ace6784563a4c8c815966f8528d814e: Generated new TSK 0
I20260806 15:07:14.453608 505 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:07:14.454074 520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:14.449581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:53165 (local address 127.0.0.1:39507)
0806 15:07:14.449616 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:14.449618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:14.450589 (+ 971us) server_negotiation.cc:407] Connection header received
0806 15:07:14.450661 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:14.450662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:14.450667 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:14.450694 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:14.453853 (+ 3159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:14.453854 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:14.453854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:07:14.453859 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:14.453874 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:07:14.453878 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:14.453948 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:14.454004 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 15:07:14.454016 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:07:14.455297 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 378
2026-08-06T15:07:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1350 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:07:14 dist-test-slave-tc2s krb5kdc[531](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:07:14 dist-test-slave-tc2s krb5kdc[531](info): set up 2 sockets
Aug 06 15:07:14 dist-test-slave-tc2s krb5kdc[531](info): commencing operation
krb5kdc: starting...
W20260806 15:07:15.568603 945 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:07:15 dist-test-slave-tc2s krb5kdc[531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:07:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:07:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:07:15.678066 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:44873
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:44575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:44873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:07:15.756767 552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:07:15.756970 552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:07:15.757002 552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:07:15.758133 552 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:07:15.758172 552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:07:15.758191 552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:07:15.758208 552 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:07:15.758224 552 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:07:15.759675 552 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:44575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:44873
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:44873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:07:15.759912 552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:07:15.760115 552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:07:15.764582 561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:15.764595 559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:15.764590 558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:15.764665 552 server_base.cc:1034] running on GCE node
I20260806 15:07:15.764983 552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:07:15.765237 552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:07:15.766371 552 hybrid_clock.cc:648] HybridClock initialized: now 1786028835766356 us; error 29 us; skew 500 ppm
Aug 06 15:07:15 dist-test-slave-tc2s krb5kdc[531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028835, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:07:15.771584 552 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:07:15.772007 552 webserver.cc:460] Webserver started at http://127.0.236.126:45839/ using document root <none> and password file <none>
I20260806 15:07:15.772209 552 fs_manager.cc:359] Metadata directory not provided
I20260806 15:07:15.772255 552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:07:15.772378 552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:07:15.773375 552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "029b6c6cb6ce4bebb3eeb8d5a05dad16"
format_stamp: "Formatted at 2026-08-06 15:07:15 on dist-test-slave-tc2s"
server_key: "b7c4ec6f009c9c9069d9714f51f7d087"
server_key_iv: "5f430870cf1ea35e5dc157e55567266c"
server_key_version: "encryptionkey@0"
I20260806 15:07:15.773687 552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "029b6c6cb6ce4bebb3eeb8d5a05dad16"
format_stamp: "Formatted at 2026-08-06 15:07:15 on dist-test-slave-tc2s"
server_key: "b7c4ec6f009c9c9069d9714f51f7d087"
server_key_iv: "5f430870cf1ea35e5dc157e55567266c"
server_key_version: "encryptionkey@0"
I20260806 15:07:15.774904 552 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:07:15.775535 568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:15.775682 552 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:07:15.775763 552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "029b6c6cb6ce4bebb3eeb8d5a05dad16"
format_stamp: "Formatted at 2026-08-06 15:07:15 on dist-test-slave-tc2s"
server_key: "b7c4ec6f009c9c9069d9714f51f7d087"
server_key_iv: "5f430870cf1ea35e5dc157e55567266c"
server_key_version: "encryptionkey@0"
I20260806 15:07:15.775825 552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:07:15.785149 552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:07:15.786751 552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:07:15.786876 552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:07:15.790632 552 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:44873
I20260806 15:07:15.790668 646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:44873 every 8 connection(s)
I20260806 15:07:15.790975 552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:07:15.791747 647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:07:15.792203 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 552
I20260806 15:07:15.792330 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:07:15.792513 945 external_mini_cluster.cc:1428] Setting key 9deec6452ab6b6ba43f35b657bddfaad
I20260806 15:07:15.794445 647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 029b6c6cb6ce4bebb3eeb8d5a05dad16: Bootstrap starting.
I20260806 15:07:15.794996 647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 029b6c6cb6ce4bebb3eeb8d5a05dad16: Neither blocks nor log segments found. Creating new log.
I20260806 15:07:15.795253 647 log.cc:824] T 00000000000000000000000000000000 P 029b6c6cb6ce4bebb3eeb8d5a05dad16: Log is configured to *not* fsync() on all Append() calls
I20260806 15:07:15.795840 647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 029b6c6cb6ce4bebb3eeb8d5a05dad16: No bootstrap required, opened a new log
I20260806 15:07:15.797628 647 raft_consensus.cc:348] T 00000000000000000000000000000000 P 029b6c6cb6ce4bebb3eeb8d5a05dad16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "029b6c6cb6ce4bebb3eeb8d5a05dad16" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44873 } }
I20260806 15:07:15.797750 647 raft_consensus.cc:374] T 00000000000000000000000000000000 P 029b6c6cb6ce4bebb3eeb8d5a05dad16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:07:15.797784 647 raft_consensus.cc:729] T 00000000000000000000000000000000 P 029b6c6cb6ce4bebb3eeb8d5a05dad16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 029b6c6cb6ce4bebb3eeb8d5a05dad16, State: Initialized, Role: FOLLOWER
I20260806 15:07:15.797930 647 consensus_queue.cc:260] T 00000000000000000000000000000000 P 029b6c6cb6ce4bebb3eeb8d5a05dad16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "029b6c6cb6ce4bebb3eeb8d5a05dad16" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44873 } }
I20260806 15:07:15.798017 647 raft_consensus.cc:388] T 00000000000000000000000000000000 P 029b6c6cb6ce4bebb3eeb8d5a05dad16 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:07:15.798048 647 raft_consensus.cc:482] T 00000000000000000000000000000000 P 029b6c6cb6ce4bebb3eeb8d5a05dad16 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:07:15.798095 647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 029b6c6cb6ce4bebb3eeb8d5a05dad16 [term 0 FOLLOWER]: Advancing to term 1
Aug 06 15:07:15 dist-test-slave-tc2s krb5kdc[531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:07:15.798717 647 raft_consensus.cc:504] T 00000000000000000000000000000000 P 029b6c6cb6ce4bebb3eeb8d5a05dad16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "029b6c6cb6ce4bebb3eeb8d5a05dad16" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44873 } }
I20260806 15:07:15.798813 647 leader_election.cc:302] T 00000000000000000000000000000000 P 029b6c6cb6ce4bebb3eeb8d5a05dad16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 029b6c6cb6ce4bebb3eeb8d5a05dad16; no voters:
I20260806 15:07:15.798964 647 leader_election.cc:288] T 00000000000000000000000000000000 P 029b6c6cb6ce4bebb3eeb8d5a05dad16 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:07:15.798993 656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 029b6c6cb6ce4bebb3eeb8d5a05dad16 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:07:15.799099 656 raft_consensus.cc:686] T 00000000000000000000000000000000 P 029b6c6cb6ce4bebb3eeb8d5a05dad16 [term 1 LEADER]: Becoming Leader. State: Replica: 029b6c6cb6ce4bebb3eeb8d5a05dad16, State: Running, Role: LEADER
I20260806 15:07:15.799213 647 sys_catalog.cc:564] T 00000000000000000000000000000000 P 029b6c6cb6ce4bebb3eeb8d5a05dad16 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:07:15.799247 656 consensus_queue.cc:237] T 00000000000000000000000000000000 P 029b6c6cb6ce4bebb3eeb8d5a05dad16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "029b6c6cb6ce4bebb3eeb8d5a05dad16" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44873 } }
I20260806 15:07:15.799844 658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 029b6c6cb6ce4bebb3eeb8d5a05dad16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "029b6c6cb6ce4bebb3eeb8d5a05dad16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "029b6c6cb6ce4bebb3eeb8d5a05dad16" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44873 } } }
I20260806 15:07:15.799925 658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 029b6c6cb6ce4bebb3eeb8d5a05dad16 [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:15.800134 656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 029b6c6cb6ce4bebb3eeb8d5a05dad16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 029b6c6cb6ce4bebb3eeb8d5a05dad16. Latest consensus state: current_term: 1 leader_uuid: "029b6c6cb6ce4bebb3eeb8d5a05dad16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "029b6c6cb6ce4bebb3eeb8d5a05dad16" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 44873 } } }
I20260806 15:07:15.800195 656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 029b6c6cb6ce4bebb3eeb8d5a05dad16 [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:15.800457 667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:07:15.801055 667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:07:15.802446 667 catalog_manager.cc:1347] Generated new cluster ID: 3f96c09fd92744b2a1076e1fbcbe53b3
I20260806 15:07:15.802491 667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:07:15.802996 652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:15.793003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60029 (local address 127.0.236.126:44873)
0806 15:07:15.793158 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:15.793167 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:15.793193 (+ 26us) server_negotiation.cc:407] Connection header received
0806 15:07:15.793575 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:15.793581 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:15.793623 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:15.793732 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:15.794507 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:15.795416 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:15.796198 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:15.796628 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:15.796728 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:15.799114 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:15.799119 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:15.799119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:15.799132 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:15.801029 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:15.801471 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:15.801474 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:15.801474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:15.801580 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:15.801812 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:15.801813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:15.801813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:15.802123 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:15.802610 (+ 487us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:15.802694 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 15:07:15.802795 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
I20260806 15:07:15.803838 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:07:15.811949 652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:15.805302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40169 (local address 127.0.236.126:44873)
0806 15:07:15.805333 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:15.805335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:15.805360 (+ 25us) server_negotiation.cc:407] Connection header received
0806 15:07:15.805429 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:15.805430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:15.805445 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:15.805487 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:15.806092 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:15.806778 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:15.807732 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:15.808051 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:15.808102 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:15.809399 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:15.809399 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:15.809400 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:15.809405 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:15.810867 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:15.811267 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:15.811267 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:15.811268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:15.811315 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:15.811505 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:15.811505 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:15.811505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:15.811606 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:15.811686 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:15.811818 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 15:07:15.811871 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:07:15.818603 667 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:07:15.819073 667 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:07:15.822211 667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 029b6c6cb6ce4bebb3eeb8d5a05dad16: Generated new TSK 0
I20260806 15:07:15.822355 667 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:07:15.824226 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 552
W20260806 15:07:15.838946 528 connection.cc:535] client connection to 127.0.236.126:44873 recv error: Network error: failed to read from TLS socket (remote: 127.0.236.126:44873): Connection reset by peer (error 104)
2026-08-06T15:07:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1371 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:07:15 dist-test-slave-tc2s krb5kdc[694](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:07:15 dist-test-slave-tc2s krb5kdc[694](info): set up 2 sockets
Aug 06 15:07:15 dist-test-slave-tc2s krb5kdc[694](info): commencing operation
krb5kdc: starting...
W20260806 15:07:16.945948 945 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:07:17 dist-test-slave-tc2s krb5kdc[694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:07:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:07:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:07:17.054134 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:35133
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:40815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:35133
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:07:17.132375 714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:07:17.132560 714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:07:17.132581 714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:07:17.133677 714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:07:17.133715 714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:07:17.133729 714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:07:17.133752 714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:07:17.133775 714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:07:17.135233 714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:40815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:35133
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:35133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:07:17.135450 714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:07:17.135637 714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:07:17.140105 727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:17.140136 714 server_base.cc:1034] running on GCE node
W20260806 15:07:17.140106 723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:17.140189 722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:17.140599 714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:07:17.140892 714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:07:17.142037 714 hybrid_clock.cc:648] HybridClock initialized: now 1786028837142023 us; error 30 us; skew 500 ppm
Aug 06 15:07:17 dist-test-slave-tc2s krb5kdc[694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028837, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:07:17.144321 714 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:07:17.144711 714 webserver.cc:460] Webserver started at http://127.0.236.126:42393/ using document root <none> and password file <none>
I20260806 15:07:17.144906 714 fs_manager.cc:359] Metadata directory not provided
I20260806 15:07:17.144960 714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:07:17.145083 714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:07:17.146099 714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "d2937d2b8b8546b7b82ef333e4afdac4"
format_stamp: "Formatted at 2026-08-06 15:07:17 on dist-test-slave-tc2s"
server_key: "6a31355ddd6587b229e05d7ae60508a1"
server_key_iv: "4e2bd45c39aede66d56ce547693e020f"
server_key_version: "encryptionkey@0"
I20260806 15:07:17.146410 714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "d2937d2b8b8546b7b82ef333e4afdac4"
format_stamp: "Formatted at 2026-08-06 15:07:17 on dist-test-slave-tc2s"
server_key: "6a31355ddd6587b229e05d7ae60508a1"
server_key_iv: "4e2bd45c39aede66d56ce547693e020f"
server_key_version: "encryptionkey@0"
I20260806 15:07:17.147598 714 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:07:17.148263 739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:17.148401 714 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:07:17.148485 714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "d2937d2b8b8546b7b82ef333e4afdac4"
format_stamp: "Formatted at 2026-08-06 15:07:17 on dist-test-slave-tc2s"
server_key: "6a31355ddd6587b229e05d7ae60508a1"
server_key_iv: "4e2bd45c39aede66d56ce547693e020f"
server_key_version: "encryptionkey@0"
I20260806 15:07:17.148543 714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:07:17.156450 714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:07:17.158388 714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:07:17.158512 714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:07:17.162235 714 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:35133
I20260806 15:07:17.162266 813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:35133 every 8 connection(s)
I20260806 15:07:17.162575 714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:07:17.163424 816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:07:17.165519 816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2937d2b8b8546b7b82ef333e4afdac4: Bootstrap starting.
I20260806 15:07:17.166074 816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2937d2b8b8546b7b82ef333e4afdac4: Neither blocks nor log segments found. Creating new log.
I20260806 15:07:17.166333 816 log.cc:824] T 00000000000000000000000000000000 P d2937d2b8b8546b7b82ef333e4afdac4: Log is configured to *not* fsync() on all Append() calls
I20260806 15:07:17.166951 816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2937d2b8b8546b7b82ef333e4afdac4: No bootstrap required, opened a new log
I20260806 15:07:17.168061 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 714
I20260806 15:07:17.168179 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:07:17.168363 945 external_mini_cluster.cc:1428] Setting key 401b1f77f74fad9803ca7750cc2f228b
I20260806 15:07:17.168718 816 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2937d2b8b8546b7b82ef333e4afdac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2937d2b8b8546b7b82ef333e4afdac4" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35133 } }
I20260806 15:07:17.168887 816 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2937d2b8b8546b7b82ef333e4afdac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:07:17.168963 816 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2937d2b8b8546b7b82ef333e4afdac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2937d2b8b8546b7b82ef333e4afdac4, State: Initialized, Role: FOLLOWER
I20260806 15:07:17.169098 816 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2937d2b8b8546b7b82ef333e4afdac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2937d2b8b8546b7b82ef333e4afdac4" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35133 } }
I20260806 15:07:17.169178 816 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2937d2b8b8546b7b82ef333e4afdac4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:07:17.169219 816 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2937d2b8b8546b7b82ef333e4afdac4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:07:17.169267 816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2937d2b8b8546b7b82ef333e4afdac4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:07:17.170181 816 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2937d2b8b8546b7b82ef333e4afdac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2937d2b8b8546b7b82ef333e4afdac4" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35133 } }
I20260806 15:07:17.170269 816 leader_election.cc:302] T 00000000000000000000000000000000 P d2937d2b8b8546b7b82ef333e4afdac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2937d2b8b8546b7b82ef333e4afdac4; no voters:
I20260806 15:07:17.170413 816 leader_election.cc:288] T 00000000000000000000000000000000 P d2937d2b8b8546b7b82ef333e4afdac4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:07:17.170463 821 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2937d2b8b8546b7b82ef333e4afdac4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:07:17.170598 821 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2937d2b8b8546b7b82ef333e4afdac4 [term 1 LEADER]: Becoming Leader. State: Replica: d2937d2b8b8546b7b82ef333e4afdac4, State: Running, Role: LEADER
I20260806 15:07:17.170661 816 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2937d2b8b8546b7b82ef333e4afdac4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:07:17.170717 821 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2937d2b8b8546b7b82ef333e4afdac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2937d2b8b8546b7b82ef333e4afdac4" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35133 } }
I20260806 15:07:17.171838 824 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2937d2b8b8546b7b82ef333e4afdac4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2937d2b8b8546b7b82ef333e4afdac4. Latest consensus state: current_term: 1 leader_uuid: "d2937d2b8b8546b7b82ef333e4afdac4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2937d2b8b8546b7b82ef333e4afdac4" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35133 } } }
I20260806 15:07:17.171913 824 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2937d2b8b8546b7b82ef333e4afdac4 [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:17.172202 823 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2937d2b8b8546b7b82ef333e4afdac4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2937d2b8b8546b7b82ef333e4afdac4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2937d2b8b8546b7b82ef333e4afdac4" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35133 } } }
I20260806 15:07:17.172340 823 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2937d2b8b8546b7b82ef333e4afdac4 [sys.catalog]: This master's current role is: LEADER
W20260806 15:07:17.172425 846 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d2937d2b8b8546b7b82ef333e4afdac4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:07:17.172468 846 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:07:17.172521 848 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:07:17.172679 848 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:07:17.173852 820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:17.168837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53655 (local address 127.0.236.126:35133)
0806 15:07:17.169029 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:17.169038 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:17.169071 (+ 33us) server_negotiation.cc:407] Connection header received
0806 15:07:17.169163 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:17.169168 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:17.169211 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:17.169340 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:17.169550 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:17.170551 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:17.171698 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:17.172297 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:17.172400 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:17.173235 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:17.173239 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:17.173239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:07:17.173401 (+ 162us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:17.173431 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:07:17.173441 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:17.173516 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:17.173615 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 15:07:17.173642 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":74,"threads_started":1}
I20260806 15:07:17.174269 848 catalog_manager.cc:1347] Generated new cluster ID: bb143d1fcdb84cf6b1cb8d9d372706ad
I20260806 15:07:17.174324 848 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:07:17.174744 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:07:17.184706 820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:17.182173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45913 (local address 127.0.236.126:35133)
0806 15:07:17.182206 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:17.182208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:17.182332 (+ 124us) server_negotiation.cc:407] Connection header received
0806 15:07:17.182385 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:17.182386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:17.182396 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:17.182458 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:17.182580 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:17.183013 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:17.183662 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:17.183860 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:17.183911 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:17.184484 (+ 573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:17.184486 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:17.184486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:07:17.184490 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:17.184504 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:07:17.184509 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:17.184550 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:17.184626 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 15:07:17.184638 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:07:17.194072 848 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:07:17.194736 848 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:07:17.199893 848 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2937d2b8b8546b7b82ef333e4afdac4: Generated new TSK 0
I20260806 15:07:17.200047 848 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:07:17.203037 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 714
W20260806 15:07:17.216871 692 connection.cc:535] client connection to 127.0.236.126:35133 recv error: Network error: failed to read from TLS socket (remote: 127.0.236.126:35133): Connection reset by peer (error 104)
2026-08-06T15:07:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1377 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:07:17 dist-test-slave-tc2s krb5kdc[868](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:07:17 dist-test-slave-tc2s krb5kdc[868](info): set up 2 sockets
Aug 06 15:07:17 dist-test-slave-tc2s krb5kdc[868](info): commencing operation
krb5kdc: starting...
W20260806 15:07:18.315526 945 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:07:18 dist-test-slave-tc2s krb5kdc[868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:07:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:07:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:07:18.425822 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:34167
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:39733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:34167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:07:18.505074 885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:07:18.505273 885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:07:18.505306 885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:07:18.506448 885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:07:18.506487 885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:07:18.506506 885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:07:18.506522 885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:07:18.506536 885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:07:18.507941 885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:39733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:34167
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:34167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:07:18.508169 885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:07:18.508373 885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:07:18.512846 893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:18.512847 899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:18.512923 885 server_base.cc:1034] running on GCE node
W20260806 15:07:18.512861 895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:18.513262 885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:07:18.513552 885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:07:18.514679 885 hybrid_clock.cc:648] HybridClock initialized: now 1786028838514663 us; error 32 us; skew 500 ppm
Aug 06 15:07:18 dist-test-slave-tc2s krb5kdc[868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028838, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:07:18.516903 885 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:07:18.517302 885 webserver.cc:460] Webserver started at http://127.0.236.126:35347/ using document root <none> and password file <none>
I20260806 15:07:18.517499 885 fs_manager.cc:359] Metadata directory not provided
I20260806 15:07:18.517547 885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:07:18.517668 885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:07:18.518666 885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "544833cd222948abb3582d4efe7e4b70"
format_stamp: "Formatted at 2026-08-06 15:07:18 on dist-test-slave-tc2s"
server_key: "4c0601471241d3411e4be91dc9f6479d"
server_key_iv: "0847f4768624c1a8efaf572d54a2c243"
server_key_version: "encryptionkey@0"
I20260806 15:07:18.518977 885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "544833cd222948abb3582d4efe7e4b70"
format_stamp: "Formatted at 2026-08-06 15:07:18 on dist-test-slave-tc2s"
server_key: "4c0601471241d3411e4be91dc9f6479d"
server_key_iv: "0847f4768624c1a8efaf572d54a2c243"
server_key_version: "encryptionkey@0"
I20260806 15:07:18.520140 885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 15:07:18.520769 909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:18.520915 885 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:07:18.520994 885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "544833cd222948abb3582d4efe7e4b70"
format_stamp: "Formatted at 2026-08-06 15:07:18 on dist-test-slave-tc2s"
server_key: "4c0601471241d3411e4be91dc9f6479d"
server_key_iv: "0847f4768624c1a8efaf572d54a2c243"
server_key_version: "encryptionkey@0"
I20260806 15:07:18.521059 885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:07:18.524160 885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:07:18.525730 885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:07:18.525851 885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:07:18.529546 885 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:34167
I20260806 15:07:18.529589 989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:34167 every 8 connection(s)
I20260806 15:07:18.529853 885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:07:18.530767 990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:07:18.532748 990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 544833cd222948abb3582d4efe7e4b70: Bootstrap starting.
I20260806 15:07:18.533249 990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 544833cd222948abb3582d4efe7e4b70: Neither blocks nor log segments found. Creating new log.
I20260806 15:07:18.533481 990 log.cc:824] T 00000000000000000000000000000000 P 544833cd222948abb3582d4efe7e4b70: Log is configured to *not* fsync() on all Append() calls
I20260806 15:07:18.534090 990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 544833cd222948abb3582d4efe7e4b70: No bootstrap required, opened a new log
I20260806 15:07:18.535579 990 raft_consensus.cc:348] T 00000000000000000000000000000000 P 544833cd222948abb3582d4efe7e4b70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "544833cd222948abb3582d4efe7e4b70" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 34167 } }
I20260806 15:07:18.535701 990 raft_consensus.cc:374] T 00000000000000000000000000000000 P 544833cd222948abb3582d4efe7e4b70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:07:18.535720 990 raft_consensus.cc:729] T 00000000000000000000000000000000 P 544833cd222948abb3582d4efe7e4b70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 544833cd222948abb3582d4efe7e4b70, State: Initialized, Role: FOLLOWER
I20260806 15:07:18.535817 990 consensus_queue.cc:260] T 00000000000000000000000000000000 P 544833cd222948abb3582d4efe7e4b70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "544833cd222948abb3582d4efe7e4b70" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 34167 } }
I20260806 15:07:18.535867 990 raft_consensus.cc:388] T 00000000000000000000000000000000 P 544833cd222948abb3582d4efe7e4b70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:07:18.535887 990 raft_consensus.cc:482] T 00000000000000000000000000000000 P 544833cd222948abb3582d4efe7e4b70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:07:18.535918 990 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 544833cd222948abb3582d4efe7e4b70 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:07:18.536339 990 raft_consensus.cc:504] T 00000000000000000000000000000000 P 544833cd222948abb3582d4efe7e4b70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "544833cd222948abb3582d4efe7e4b70" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 34167 } }
I20260806 15:07:18.536423 990 leader_election.cc:302] T 00000000000000000000000000000000 P 544833cd222948abb3582d4efe7e4b70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 544833cd222948abb3582d4efe7e4b70; no voters:
I20260806 15:07:18.536536 990 leader_election.cc:288] T 00000000000000000000000000000000 P 544833cd222948abb3582d4efe7e4b70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:07:18.536577 995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 544833cd222948abb3582d4efe7e4b70 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:07:18.536739 990 sys_catalog.cc:564] T 00000000000000000000000000000000 P 544833cd222948abb3582d4efe7e4b70 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:07:18.536755 995 raft_consensus.cc:686] T 00000000000000000000000000000000 P 544833cd222948abb3582d4efe7e4b70 [term 1 LEADER]: Becoming Leader. State: Replica: 544833cd222948abb3582d4efe7e4b70, State: Running, Role: LEADER
I20260806 15:07:18.536916 995 consensus_queue.cc:237] T 00000000000000000000000000000000 P 544833cd222948abb3582d4efe7e4b70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "544833cd222948abb3582d4efe7e4b70" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 34167 } }
I20260806 15:07:18.537465 998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 544833cd222948abb3582d4efe7e4b70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 544833cd222948abb3582d4efe7e4b70. Latest consensus state: current_term: 1 leader_uuid: "544833cd222948abb3582d4efe7e4b70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "544833cd222948abb3582d4efe7e4b70" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 34167 } } }
I20260806 15:07:18.537477 996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 544833cd222948abb3582d4efe7e4b70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "544833cd222948abb3582d4efe7e4b70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "544833cd222948abb3582d4efe7e4b70" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 34167 } } }
I20260806 15:07:18.537555 998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 544833cd222948abb3582d4efe7e4b70 [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:18.537571 996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 544833cd222948abb3582d4efe7e4b70 [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:18.537755 1006 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:07:18.538237 1006 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:07:18.539652 1006 catalog_manager.cc:1347] Generated new cluster ID: 5554e43b9716465abb07074b38bf1c58
I20260806 15:07:18.539695 1006 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:07:18.539749 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 885
I20260806 15:07:18.539877 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:07:18.540071 945 external_mini_cluster.cc:1428] Setting key 662c2b6d386bf96b3461c337e3dc6db7
I20260806 15:07:18.546803 1026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:18.540587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42705 (local address 127.0.236.126:34167)
0806 15:07:18.541996 (+ 1409us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:18.542021 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:18.542051 (+ 30us) server_negotiation.cc:407] Connection header received
0806 15:07:18.542149 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:18.542155 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:18.542181 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:18.542297 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:18.546377 (+ 4080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:18.546381 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:18.546382 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:07:18.546386 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:18.546439 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:07:18.546454 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:18.546523 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:18.546580 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 15:07:18.546618 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1365,"thread_start_us":60,"threads_started":1}
I20260806 15:07:18.547729 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:07:18.549991 1026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:18.549082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47091 (local address 127.0.236.126:34167)
0806 15:07:18.549114 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:18.549115 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:18.549168 (+ 53us) server_negotiation.cc:407] Connection header received
0806 15:07:18.549223 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:18.549223 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:18.549228 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:18.549283 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:18.549785 (+ 502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:18.549786 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:18.549787 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:07:18.549790 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:18.549804 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:07:18.549808 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:18.549862 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:18.549920 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 15:07:18.549932 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 15:07:18.565146 1006 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:07:18.565282 1006 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:07:18.577291 1006 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 544833cd222948abb3582d4efe7e4b70: Generated new TSK 0
I20260806 15:07:18.577453 1006 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:07:18.581643 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 885
2026-08-06T15:07:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1387 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:07:18 dist-test-slave-tc2s krb5kdc[1037](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:07:18 dist-test-slave-tc2s krb5kdc[1037](info): set up 2 sockets
Aug 06 15:07:18 dist-test-slave-tc2s krb5kdc[1037](info): commencing operation
krb5kdc: starting...
W20260806 15:07:19.703042 945 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:07:19 dist-test-slave-tc2s krb5kdc[1037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:07:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:07:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:07:19.811548 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:42097
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:34329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:42097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:07:19.889880 1059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:07:19.890089 1059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:07:19.890120 1059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:07:19.891237 1059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:07:19.891278 1059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:07:19.891299 1059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:07:19.891314 1059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:07:19.891341 1059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:07:19.892777 1059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:34329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:42097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:42097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:07:19.893005 1059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:07:19.893199 1059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 15:07:19.897778 1059 server_base.cc:1034] running on GCE node
W20260806 15:07:19.897727 1070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:19.897749 1065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:19.897734 1066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:19.898185 1059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:07:19.898466 1059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:07:19.899605 1059 hybrid_clock.cc:648] HybridClock initialized: now 1786028839899591 us; error 27 us; skew 500 ppm
Aug 06 15:07:19 dist-test-slave-tc2s krb5kdc[1037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028839, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:07:19.902026 1059 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:07:19.902448 1059 webserver.cc:460] Webserver started at http://127.0.236.126:33147/ using document root <none> and password file <none>
I20260806 15:07:19.902616 1059 fs_manager.cc:359] Metadata directory not provided
I20260806 15:07:19.902653 1059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:07:19.902742 1059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:07:19.903681 1059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "2176de7437714ec9af6a770d4fac92ce"
format_stamp: "Formatted at 2026-08-06 15:07:19 on dist-test-slave-tc2s"
server_key: "f57d1aa018a445aca531bf19c919c163"
server_key_iv: "9f8bbb2a2499b2cd2205beeb7d8b2482"
server_key_version: "encryptionkey@0"
I20260806 15:07:19.903962 1059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "2176de7437714ec9af6a770d4fac92ce"
format_stamp: "Formatted at 2026-08-06 15:07:19 on dist-test-slave-tc2s"
server_key: "f57d1aa018a445aca531bf19c919c163"
server_key_iv: "9f8bbb2a2499b2cd2205beeb7d8b2482"
server_key_version: "encryptionkey@0"
I20260806 15:07:19.905109 1059 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:07:19.905751 1078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:19.905920 1059 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:07:19.906002 1059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "2176de7437714ec9af6a770d4fac92ce"
format_stamp: "Formatted at 2026-08-06 15:07:19 on dist-test-slave-tc2s"
server_key: "f57d1aa018a445aca531bf19c919c163"
server_key_iv: "9f8bbb2a2499b2cd2205beeb7d8b2482"
server_key_version: "encryptionkey@0"
I20260806 15:07:19.906055 1059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:07:19.916827 1059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:07:19.918493 1059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:07:19.918596 1059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:07:19.922305 1059 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:42097
I20260806 15:07:19.922343 1152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:42097 every 8 connection(s)
I20260806 15:07:19.922631 1059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:07:19.923452 1154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:07:19.925302 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 1059
I20260806 15:07:19.925437 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:07:19.925642 945 external_mini_cluster.cc:1428] Setting key df57308a328e6f868f1b9533e333eb49
I20260806 15:07:19.925818 1154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2176de7437714ec9af6a770d4fac92ce: Bootstrap starting.
I20260806 15:07:19.926558 1154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2176de7437714ec9af6a770d4fac92ce: Neither blocks nor log segments found. Creating new log.
I20260806 15:07:19.926898 1154 log.cc:824] T 00000000000000000000000000000000 P 2176de7437714ec9af6a770d4fac92ce: Log is configured to *not* fsync() on all Append() calls
I20260806 15:07:19.927585 1154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2176de7437714ec9af6a770d4fac92ce: No bootstrap required, opened a new log
I20260806 15:07:19.929762 1154 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2176de7437714ec9af6a770d4fac92ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2176de7437714ec9af6a770d4fac92ce" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 42097 } }
I20260806 15:07:19.929939 1154 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2176de7437714ec9af6a770d4fac92ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:07:19.929986 1154 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2176de7437714ec9af6a770d4fac92ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2176de7437714ec9af6a770d4fac92ce, State: Initialized, Role: FOLLOWER
I20260806 15:07:19.930099 1154 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2176de7437714ec9af6a770d4fac92ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2176de7437714ec9af6a770d4fac92ce" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 42097 } }
I20260806 15:07:19.930168 1154 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2176de7437714ec9af6a770d4fac92ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:07:19.930204 1154 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2176de7437714ec9af6a770d4fac92ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:07:19.930248 1154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2176de7437714ec9af6a770d4fac92ce [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:07:19.930859 1154 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2176de7437714ec9af6a770d4fac92ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2176de7437714ec9af6a770d4fac92ce" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 42097 } }
I20260806 15:07:19.930960 1154 leader_election.cc:302] T 00000000000000000000000000000000 P 2176de7437714ec9af6a770d4fac92ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2176de7437714ec9af6a770d4fac92ce; no voters:
I20260806 15:07:19.931116 1154 leader_election.cc:288] T 00000000000000000000000000000000 P 2176de7437714ec9af6a770d4fac92ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:07:19.931159 1159 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2176de7437714ec9af6a770d4fac92ce [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:07:19.931334 1159 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2176de7437714ec9af6a770d4fac92ce [term 1 LEADER]: Becoming Leader. State: Replica: 2176de7437714ec9af6a770d4fac92ce, State: Running, Role: LEADER
I20260806 15:07:19.931365 1154 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2176de7437714ec9af6a770d4fac92ce [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:07:19.931444 1159 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2176de7437714ec9af6a770d4fac92ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2176de7437714ec9af6a770d4fac92ce" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 42097 } }
I20260806 15:07:19.931759 1159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2176de7437714ec9af6a770d4fac92ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2176de7437714ec9af6a770d4fac92ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2176de7437714ec9af6a770d4fac92ce" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 42097 } } }
I20260806 15:07:19.931842 1159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2176de7437714ec9af6a770d4fac92ce [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:19.931964 1159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2176de7437714ec9af6a770d4fac92ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2176de7437714ec9af6a770d4fac92ce. Latest consensus state: current_term: 1 leader_uuid: "2176de7437714ec9af6a770d4fac92ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2176de7437714ec9af6a770d4fac92ce" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 42097 } } }
I20260806 15:07:19.932019 1159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2176de7437714ec9af6a770d4fac92ce [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:19.932133 1171 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 06 15:07:19 dist-test-slave-tc2s krb5kdc[1037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:07:19.932617 1171 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:07:19.935122 1171 catalog_manager.cc:1347] Generated new cluster ID: 1660091116d34effb9c5708ffd0a372b
I20260806 15:07:19.935174 1171 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:07:19.936427 1157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:19.926132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55317 (local address 127.0.236.126:42097)
0806 15:07:19.926424 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:19.926432 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:19.926464 (+ 32us) server_negotiation.cc:407] Connection header received
0806 15:07:19.926997 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:19.927005 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:19.927067 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:19.927198 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:19.927822 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:19.929198 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:19.929918 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:19.930514 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:19.930626 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:19.933108 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:19.933112 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:19.933112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:19.933123 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:19.934991 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:19.935323 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:19.935326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:19.935326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:19.935429 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:19.935605 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:19.935606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:19.935606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:19.935833 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:19.935981 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:19.936096 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 15:07:19.936224 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":70,"threads_started":1}
I20260806 15:07:19.937291 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:07:19.947186 1157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:19.938536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38519 (local address 127.0.236.126:42097)
0806 15:07:19.938567 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:19.938569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:19.942136 (+ 3567us) server_negotiation.cc:407] Connection header received
0806 15:07:19.942190 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:19.942191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:19.942200 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:19.942244 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:19.942695 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:19.943146 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:19.943779 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:19.944002 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:19.944055 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:19.945047 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:19.945048 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:19.945048 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:19.945055 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:19.946255 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:19.946550 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:19.946551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:19.946551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:19.946604 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:19.946760 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:19.946761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:19.946762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:19.946861 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:19.946945 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:19.947042 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 15:07:19.947107 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:07:19.951020 1171 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:07:19.951474 1171 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:07:19.966228 1171 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2176de7437714ec9af6a770d4fac92ce: Generated new TSK 0
I20260806 15:07:19.966379 1171 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:07:19.976121 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 1059
2026-08-06T15:07:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1383 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:07:20 dist-test-slave-tc2s krb5kdc[1191](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:07:20 dist-test-slave-tc2s krb5kdc[1191](info): set up 2 sockets
Aug 06 15:07:20 dist-test-slave-tc2s krb5kdc[1191](info): commencing operation
krb5kdc: starting...
W20260806 15:07:21.084007 945 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:07:21 dist-test-slave-tc2s krb5kdc[1191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:07:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:07:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:07:21.196224 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:35107
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:43397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:35107
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:07:21.275588 1212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:07:21.275784 1212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:07:21.275815 1212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:07:21.277062 1212 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:07:21.277113 1212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:07:21.277133 1212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:07:21.277151 1212 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:07:21.277168 1212 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:07:21.278643 1212 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:43397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:35107
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:35107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:07:21.278862 1212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:07:21.279072 1212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:07:21.283514 1224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:21.283557 1219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:21.283520 1221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:21.283555 1212 server_base.cc:1034] running on GCE node
I20260806 15:07:21.283912 1212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:07:21.284157 1212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:07:21.285287 1212 hybrid_clock.cc:648] HybridClock initialized: now 1786028841285276 us; error 31 us; skew 500 ppm
Aug 06 15:07:21 dist-test-slave-tc2s krb5kdc[1191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028841, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:07:21.290601 1212 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:07:21.290985 1212 webserver.cc:460] Webserver started at http://127.0.236.126:37189/ using document root <none> and password file <none>
I20260806 15:07:21.291172 1212 fs_manager.cc:359] Metadata directory not provided
I20260806 15:07:21.291219 1212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:07:21.291343 1212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:07:21.292317 1212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "05f4363172cc4aceb70a0bb05bc63182"
format_stamp: "Formatted at 2026-08-06 15:07:21 on dist-test-slave-tc2s"
server_key: "9b588ca19f6000ed6f445300eb0a140d"
server_key_iv: "6f0017f6dbedfae9bf9f21ee8b34d9c2"
server_key_version: "encryptionkey@0"
I20260806 15:07:21.292639 1212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "05f4363172cc4aceb70a0bb05bc63182"
format_stamp: "Formatted at 2026-08-06 15:07:21 on dist-test-slave-tc2s"
server_key: "9b588ca19f6000ed6f445300eb0a140d"
server_key_iv: "6f0017f6dbedfae9bf9f21ee8b34d9c2"
server_key_version: "encryptionkey@0"
I20260806 15:07:21.293845 1212 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 15:07:21.294552 1238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:21.294703 1212 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:07:21.294785 1212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "05f4363172cc4aceb70a0bb05bc63182"
format_stamp: "Formatted at 2026-08-06 15:07:21 on dist-test-slave-tc2s"
server_key: "9b588ca19f6000ed6f445300eb0a140d"
server_key_iv: "6f0017f6dbedfae9bf9f21ee8b34d9c2"
server_key_version: "encryptionkey@0"
I20260806 15:07:21.294844 1212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:07:21.316802 1212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:07:21.318504 1212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:07:21.318629 1212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:07:21.322510 1212 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:35107
I20260806 15:07:21.322553 1315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:35107 every 8 connection(s)
I20260806 15:07:21.322855 1212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:07:21.323657 1316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:07:21.325852 1316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05f4363172cc4aceb70a0bb05bc63182: Bootstrap starting.
I20260806 15:07:21.326436 1316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05f4363172cc4aceb70a0bb05bc63182: Neither blocks nor log segments found. Creating new log.
I20260806 15:07:21.326668 1316 log.cc:824] T 00000000000000000000000000000000 P 05f4363172cc4aceb70a0bb05bc63182: Log is configured to *not* fsync() on all Append() calls
I20260806 15:07:21.327272 1316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05f4363172cc4aceb70a0bb05bc63182: No bootstrap required, opened a new log
I20260806 15:07:21.328816 1316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05f4363172cc4aceb70a0bb05bc63182 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05f4363172cc4aceb70a0bb05bc63182" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35107 } }
I20260806 15:07:21.328948 1316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05f4363172cc4aceb70a0bb05bc63182 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:07:21.328974 1316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05f4363172cc4aceb70a0bb05bc63182 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05f4363172cc4aceb70a0bb05bc63182, State: Initialized, Role: FOLLOWER
I20260806 15:07:21.329079 1316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05f4363172cc4aceb70a0bb05bc63182 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05f4363172cc4aceb70a0bb05bc63182" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35107 } }
I20260806 15:07:21.329155 1316 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05f4363172cc4aceb70a0bb05bc63182 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:07:21.329183 1316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05f4363172cc4aceb70a0bb05bc63182 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:07:21.329214 1316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05f4363172cc4aceb70a0bb05bc63182 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:07:21.329679 1316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05f4363172cc4aceb70a0bb05bc63182 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05f4363172cc4aceb70a0bb05bc63182" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35107 } }
I20260806 15:07:21.329766 1316 leader_election.cc:302] T 00000000000000000000000000000000 P 05f4363172cc4aceb70a0bb05bc63182 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05f4363172cc4aceb70a0bb05bc63182; no voters:
I20260806 15:07:21.329885 1316 leader_election.cc:288] T 00000000000000000000000000000000 P 05f4363172cc4aceb70a0bb05bc63182 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:07:21.329957 1319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05f4363172cc4aceb70a0bb05bc63182 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:07:21.330120 1319 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05f4363172cc4aceb70a0bb05bc63182 [term 1 LEADER]: Becoming Leader. State: Replica: 05f4363172cc4aceb70a0bb05bc63182, State: Running, Role: LEADER
I20260806 15:07:21.330147 1316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05f4363172cc4aceb70a0bb05bc63182 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:07:21.330365 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 1212
I20260806 15:07:21.330475 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:07:21.330631 945 external_mini_cluster.cc:1428] Setting key b172a68bb54a2ac7456e792ac1203e27
I20260806 15:07:21.330209 1319 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05f4363172cc4aceb70a0bb05bc63182 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05f4363172cc4aceb70a0bb05bc63182" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35107 } }
I20260806 15:07:21.331339 1321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05f4363172cc4aceb70a0bb05bc63182 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05f4363172cc4aceb70a0bb05bc63182. Latest consensus state: current_term: 1 leader_uuid: "05f4363172cc4aceb70a0bb05bc63182" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05f4363172cc4aceb70a0bb05bc63182" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35107 } } }
I20260806 15:07:21.331449 1321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05f4363172cc4aceb70a0bb05bc63182 [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:21.331738 1320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05f4363172cc4aceb70a0bb05bc63182 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05f4363172cc4aceb70a0bb05bc63182" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05f4363172cc4aceb70a0bb05bc63182" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 35107 } } }
W20260806 15:07:21.331957 1343 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 05f4363172cc4aceb70a0bb05bc63182: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 15:07:21.332062 1320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05f4363172cc4aceb70a0bb05bc63182 [sys.catalog]: This master's current role is: LEADER
W20260806 15:07:21.332073 1343 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:07:21.332144 1344 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:07:21.332304 1344 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:07:21.333775 1344 catalog_manager.cc:1347] Generated new cluster ID: ff627b07c1d04660b36f59789c106085
I20260806 15:07:21.333828 1344 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:07:21.336848 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:21.331060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34477 (local address 127.0.236.126:35107)
0806 15:07:21.331836 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:21.331845 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:21.331878 (+ 33us) server_negotiation.cc:407] Connection header received
0806 15:07:21.332082 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:21.332090 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:21.332149 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:21.332292 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:21.332515 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:21.333676 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:21.334830 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:21.335439 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:21.335535 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:21.336228 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:21.336232 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:21.336232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:07:21.336415 (+ 183us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:21.336453 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:07:21.336466 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:21.336522 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:21.336599 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 15:07:21.336657 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":729,"thread_start_us":72,"threads_started":1}
I20260806 15:07:21.337656 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:07:21.342307 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:21.339020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47497 (local address 127.0.236.126:35107)
0806 15:07:21.339048 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:21.339050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:21.339165 (+ 115us) server_negotiation.cc:407] Connection header received
0806 15:07:21.339240 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:21.339241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:21.339249 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:21.339274 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:21.339437 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:21.340143 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:21.341113 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:21.341397 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:21.341429 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:21.342094 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:21.342096 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:21.342096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:07:21.342100 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:21.342111 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:07:21.342115 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:21.342138 (+ 23us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:21.342220 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 15:07:21.342242 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 15:07:21.345193 1344 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:07:21.345666 1344 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:07:21.352161 1344 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05f4363172cc4aceb70a0bb05bc63182: Generated new TSK 0
I20260806 15:07:21.352305 1344 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:07:21.357852 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 1212
2026-08-06T15:07:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1382 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:07:21 dist-test-slave-tc2s krb5kdc[1367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:07:21 dist-test-slave-tc2s krb5kdc[1367](info): set up 2 sockets
Aug 06 15:07:21 dist-test-slave-tc2s krb5kdc[1367](info): commencing operation
krb5kdc: starting...
W20260806 15:07:22.473685 945 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:07:22 dist-test-slave-tc2s krb5kdc[1367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:07:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:07:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:07:22.584262 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:37403
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:38399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:37403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:07:22.663762 1389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:07:22.663951 1389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:07:22.663972 1389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:07:22.665076 1389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:07:22.665115 1389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:07:22.665128 1389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:07:22.665141 1389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:07:22.665163 1389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:07:22.666642 1389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:38399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:37403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:37403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:07:22.666857 1389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:07:22.667057 1389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 15:07:22.671633 1389 server_base.cc:1034] running on GCE node
W20260806 15:07:22.671583 1395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:22.671574 1396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:22.671633 1400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:22.672104 1389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:07:22.672369 1389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:07:22.673507 1389 hybrid_clock.cc:648] HybridClock initialized: now 1786028842673493 us; error 37 us; skew 500 ppm
Aug 06 15:07:22 dist-test-slave-tc2s krb5kdc[1367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028842, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:07:22.675789 1389 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:07:22.676203 1389 webserver.cc:460] Webserver started at http://127.0.236.126:38665/ using document root <none> and password file <none>
I20260806 15:07:22.676411 1389 fs_manager.cc:359] Metadata directory not provided
I20260806 15:07:22.676460 1389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:07:22.676584 1389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:07:22.677544 1389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "89c3ae134ab641d4a4e7b80984df2ccb"
format_stamp: "Formatted at 2026-08-06 15:07:22 on dist-test-slave-tc2s"
server_key: "149279820a45c6b202613e8ba8b83f57"
server_key_iv: "2387dd9fc2c1ab0fe43f1413d129a881"
server_key_version: "encryptionkey@0"
I20260806 15:07:22.677846 1389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "89c3ae134ab641d4a4e7b80984df2ccb"
format_stamp: "Formatted at 2026-08-06 15:07:22 on dist-test-slave-tc2s"
server_key: "149279820a45c6b202613e8ba8b83f57"
server_key_iv: "2387dd9fc2c1ab0fe43f1413d129a881"
server_key_version: "encryptionkey@0"
I20260806 15:07:22.679057 1389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:07:22.679713 1409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:22.679860 1389 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:07:22.679948 1389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "89c3ae134ab641d4a4e7b80984df2ccb"
format_stamp: "Formatted at 2026-08-06 15:07:22 on dist-test-slave-tc2s"
server_key: "149279820a45c6b202613e8ba8b83f57"
server_key_iv: "2387dd9fc2c1ab0fe43f1413d129a881"
server_key_version: "encryptionkey@0"
I20260806 15:07:22.680008 1389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:07:22.683172 1389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:07:22.684777 1389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:07:22.684897 1389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:07:22.688668 1389 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:37403
I20260806 15:07:22.688709 1476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:37403 every 8 connection(s)
I20260806 15:07:22.688988 1389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:07:22.689874 1478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:07:22.691980 1478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89c3ae134ab641d4a4e7b80984df2ccb: Bootstrap starting.
I20260806 15:07:22.692504 1478 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89c3ae134ab641d4a4e7b80984df2ccb: Neither blocks nor log segments found. Creating new log.
I20260806 15:07:22.692739 1478 log.cc:824] T 00000000000000000000000000000000 P 89c3ae134ab641d4a4e7b80984df2ccb: Log is configured to *not* fsync() on all Append() calls
I20260806 15:07:22.693305 1478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89c3ae134ab641d4a4e7b80984df2ccb: No bootstrap required, opened a new log
I20260806 15:07:22.694885 1478 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89c3ae134ab641d4a4e7b80984df2ccb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89c3ae134ab641d4a4e7b80984df2ccb" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 37403 } }
I20260806 15:07:22.695013 1478 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89c3ae134ab641d4a4e7b80984df2ccb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:07:22.695041 1478 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89c3ae134ab641d4a4e7b80984df2ccb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89c3ae134ab641d4a4e7b80984df2ccb, State: Initialized, Role: FOLLOWER
I20260806 15:07:22.695144 1478 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89c3ae134ab641d4a4e7b80984df2ccb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89c3ae134ab641d4a4e7b80984df2ccb" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 37403 } }
I20260806 15:07:22.695216 1478 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89c3ae134ab641d4a4e7b80984df2ccb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:07:22.695243 1478 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89c3ae134ab641d4a4e7b80984df2ccb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:07:22.695273 1478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89c3ae134ab641d4a4e7b80984df2ccb [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:07:22.695729 1478 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89c3ae134ab641d4a4e7b80984df2ccb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89c3ae134ab641d4a4e7b80984df2ccb" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 37403 } }
I20260806 15:07:22.695811 1478 leader_election.cc:302] T 00000000000000000000000000000000 P 89c3ae134ab641d4a4e7b80984df2ccb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89c3ae134ab641d4a4e7b80984df2ccb; no voters:
I20260806 15:07:22.695927 1478 leader_election.cc:288] T 00000000000000000000000000000000 P 89c3ae134ab641d4a4e7b80984df2ccb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:07:22.695981 1482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89c3ae134ab641d4a4e7b80984df2ccb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:07:22.696146 1478 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89c3ae134ab641d4a4e7b80984df2ccb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:07:22.696321 1482 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89c3ae134ab641d4a4e7b80984df2ccb [term 1 LEADER]: Becoming Leader. State: Replica: 89c3ae134ab641d4a4e7b80984df2ccb, State: Running, Role: LEADER
I20260806 15:07:22.696424 1482 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89c3ae134ab641d4a4e7b80984df2ccb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89c3ae134ab641d4a4e7b80984df2ccb" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 37403 } }
I20260806 15:07:22.696738 1483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89c3ae134ab641d4a4e7b80984df2ccb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89c3ae134ab641d4a4e7b80984df2ccb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89c3ae134ab641d4a4e7b80984df2ccb" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 37403 } } }
I20260806 15:07:22.696750 1484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89c3ae134ab641d4a4e7b80984df2ccb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89c3ae134ab641d4a4e7b80984df2ccb. Latest consensus state: current_term: 1 leader_uuid: "89c3ae134ab641d4a4e7b80984df2ccb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89c3ae134ab641d4a4e7b80984df2ccb" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 37403 } } }
I20260806 15:07:22.696832 1483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89c3ae134ab641d4a4e7b80984df2ccb [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:22.696835 1484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89c3ae134ab641d4a4e7b80984df2ccb [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:22.697067 1492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:07:22.697572 1492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:07:22.698786 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 1389
I20260806 15:07:22.698902 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:07:22.699096 945 external_mini_cluster.cc:1428] Setting key 3eb853a8206fec98284b14a18292157d
I20260806 15:07:22.699167 1492 catalog_manager.cc:1347] Generated new cluster ID: 349dcb081acb40b8891a557a2e7e7a0f
I20260806 15:07:22.699220 1492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:07:22.703401 1507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:22.699514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59985 (local address 127.0.236.126:37403)
0806 15:07:22.699656 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:22.699665 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:22.702090 (+ 2425us) server_negotiation.cc:407] Connection header received
0806 15:07:22.702223 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:22.702229 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:22.702251 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:22.702360 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:22.703008 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:22.703012 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:22.703012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:07:22.703016 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:22.703052 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:07:22.703062 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:22.703114 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:22.703183 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 15:07:22.703223 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":69,"threads_started":1}
I20260806 15:07:22.704392 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:07:22.710884 1507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:22.705647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40919 (local address 127.0.236.126:37403)
0806 15:07:22.705680 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:22.705682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:22.710072 (+ 4390us) server_negotiation.cc:407] Connection header received
0806 15:07:22.710130 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:22.710131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:22.710136 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:22.710176 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:22.710681 (+ 505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:22.710682 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:22.710682 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:07:22.710687 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:22.710702 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:07:22.710706 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:22.710755 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:22.710812 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 15:07:22.710823 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:07:22.733675 1492 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:07:22.733837 1492 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:07:22.737316 1492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89c3ae134ab641d4a4e7b80984df2ccb: Generated new TSK 0
I20260806 15:07:22.737491 1492 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:07:22.742394 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 1389
2026-08-06T15:07:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1387 ms)
[----------] 18 tests from AuthTokenIssuingTest (24823 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:07:22 dist-test-slave-tc2s krb5kdc[1520](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:07:22 dist-test-slave-tc2s krb5kdc[1520](info): set up 2 sockets
Aug 06 15:07:22 dist-test-slave-tc2s krb5kdc[1520](info): commencing operation
krb5kdc: starting...
W20260806 15:07:23.860664 945 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:07:23 dist-test-slave-tc2s krb5kdc[1520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:07:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:07:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:07:23.968124 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:41749
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:44645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:41749,127.0.236.125:37281,127.0.236.124:41821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:07:24.045781 1544 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 15:07:24.045995 1544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:07:24.046013 1544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:07:24.046034 1544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:07:24.047118 1544 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:07:24.047158 1544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:07:24.047170 1544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:07:24.047194 1544 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:07:24.047216 1544 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:07:24.048640 1544 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:44645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:41749,127.0.236.125:37281,127.0.236.124:41821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:41749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:07:24.048852 1544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:07:24.049060 1544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:07:24.053489 1549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:24.053541 1553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:24.053548 1544 server_base.cc:1034] running on GCE node
W20260806 15:07:24.053488 1550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:24.054047 1544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:07:24.054291 1544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:07:24.055421 1544 hybrid_clock.cc:648] HybridClock initialized: now 1786028844055402 us; error 32 us; skew 500 ppm
Aug 06 15:07:24 dist-test-slave-tc2s krb5kdc[1520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:07:24.057742 1544 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:07:24.058159 1544 webserver.cc:460] Webserver started at http://127.0.236.126:46279/ using document root <none> and password file <none>
I20260806 15:07:24.058341 1544 fs_manager.cc:359] Metadata directory not provided
I20260806 15:07:24.058389 1544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:07:24.058512 1544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:07:24.059448 1544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "ae6eeacae6e0467da80ce2b75d92fc42"
format_stamp: "Formatted at 2026-08-06 15:07:24 on dist-test-slave-tc2s"
server_key: "c36ef3f1b8609d0abfd4f44c35b935df"
server_key_iv: "863cfc0aebd599cba9b3b3e6de632168"
server_key_version: "encryptionkey@0"
I20260806 15:07:24.059754 1544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "ae6eeacae6e0467da80ce2b75d92fc42"
format_stamp: "Formatted at 2026-08-06 15:07:24 on dist-test-slave-tc2s"
server_key: "c36ef3f1b8609d0abfd4f44c35b935df"
server_key_iv: "863cfc0aebd599cba9b3b3e6de632168"
server_key_version: "encryptionkey@0"
I20260806 15:07:24.060928 1544 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:07:24.061523 1560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:24.061692 1544 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:07:24.061782 1544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "ae6eeacae6e0467da80ce2b75d92fc42"
format_stamp: "Formatted at 2026-08-06 15:07:24 on dist-test-slave-tc2s"
server_key: "c36ef3f1b8609d0abfd4f44c35b935df"
server_key_iv: "863cfc0aebd599cba9b3b3e6de632168"
server_key_version: "encryptionkey@0"
I20260806 15:07:24.061856 1544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:07:24.089401 1544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:07:24.091305 1544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:07:24.091426 1544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:07:24.095192 1544 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:41749
I20260806 15:07:24.095235 1642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:41749 every 8 connection(s)
I20260806 15:07:24.095556 1544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:07:24.096494 1643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:07:24.098007 1643 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41749 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:07:24.102180 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 1544
I20260806 15:07:24.102300 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:07:24.102489 945 external_mini_cluster.cc:1428] Setting key e944d9db924ab72095fede661f931ff5
Aug 06 15:07:24 dist-test-slave-tc2s krb5kdc[1520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:07:24.107061 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.098616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:41749 (local address 127.0.0.1:39572)
0806 15:07:24.098771 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:24.098811 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 15:07:24.099255 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:07:24.099483 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:07:24.099485 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:07:24.099721 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:07:24.099872 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:24.099875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.100834 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.100835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:24.101513 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:24.101515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.101989 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.101990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:24.102060 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:24.102550 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:07:24.102560 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:07:24.103980 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:07:24.105664 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:24.105670 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:24.105672 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:24.105855 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:24.106114 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:24.106115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:24.106115 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:24.106193 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:24.106615 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:07:24.106618 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:07:24.106667 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 15:07:24.106730 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 15:07:24.106878 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":66,"threads_started":1}
I20260806 15:07:24.107070 1648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.098664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39572 (local address 127.0.236.126:41749)
0806 15:07:24.098831 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.098835 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.098876 (+ 41us) server_negotiation.cc:407] Connection header received
0806 15:07:24.099372 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.099374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.099390 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.099465 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.099941 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.100752 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.101582 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.101880 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.102000 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:24.104055 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:24.104062 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:24.104062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:24.104068 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:24.105590 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.105950 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.105953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.105953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.106056 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.106243 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.106243 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.106244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.106549 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:24.106642 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:24.106792 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 15:07:24.106956 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":137,"thread_start_us":88,"threads_started":1}
I20260806 15:07:24.107972 1643 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.125" port: 37281 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:07:24.108246 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.108152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:37281 (local address 127.0.0.1:33126)
0806 15:07:24.108189 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:24.108229 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.125:37281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 15:07:24.108356 1568 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.236.125:37281: connect: Connection refused (error 111)
I20260806 15:07:24.108709 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.108630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:37281 (local address 127.0.0.1:33130)
0806 15:07:24.108666 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:24.108689 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.125:37281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 15:07:24.108801 1643 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.236.125:37281: Network error: Client connection negotiation failed: client connection to 127.0.236.125:37281: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.0.236.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.125.keytab.
Entry for principal kudu/127.0.236.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.125.keytab.
I20260806 15:07:24.126049 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.125:37281
--webserver_interface=127.0.236.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.125.keytab
--principal=kudu/127.0.236.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:44645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:41749,127.0.236.125:37281,127.0.236.124:41821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:07:24.158022 1643 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.236.125" port: 37281 } attempt: 1
I20260806 15:07:24.158469 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.158336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:37281 (local address 127.0.0.1:33146)
0806 15:07:24.158395 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:24.158437 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.125:37281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 15:07:24.158795 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.158715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:37281 (local address 127.0.0.1:33148)
0806 15:07:24.158752 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:24.158777 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.125:37281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 15:07:24.158877 1643 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.236.125:37281: Network error: Client connection negotiation failed: client connection to 127.0.236.125:37281: connect: Connection refused (error 111)
W20260806 15:07:24.204839 1654 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 15:07:24.205036 1654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:07:24.205053 1654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:07:24.205070 1654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:07:24.206143 1654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:07:24.206177 1654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:07:24.206190 1654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:07:24.206200 1654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:07:24.206216 1654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:07:24.207620 1654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:44645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:41749,127.0.236.125:37281,127.0.236.124:41821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.125.keytab
--principal=kudu/127.0.236.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.125:37281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.236.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:07:24.207796 1654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:07:24.207971 1654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:07:24.212301 1661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:24.212288 1662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:24.212455 1654 server_base.cc:1034] running on GCE node
W20260806 15:07:24.212577 1665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:24.212805 1654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:07:24.213064 1654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:07:24.214198 1654 hybrid_clock.cc:648] HybridClock initialized: now 1786028844214185 us; error 28 us; skew 500 ppm
Aug 06 15:07:24 dist-test-slave-tc2s krb5kdc[1520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:07:24.216542 1654 init.cc:377] Logged in from keytab as kudu/127.0.236.125@KRBTEST.COM (short username kudu)
I20260806 15:07:24.216924 1654 webserver.cc:460] Webserver started at http://127.0.236.125:37489/ using document root <none> and password file <none>
I20260806 15:07:24.217132 1654 fs_manager.cc:359] Metadata directory not provided
I20260806 15:07:24.217178 1654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:07:24.217300 1654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:07:24.218289 1654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-1/data/instance:
uuid: "48128ae3732b492197c9dda010961500"
format_stamp: "Formatted at 2026-08-06 15:07:24 on dist-test-slave-tc2s"
server_key: "c0cb0d34805b6b981e8117627758ceb8"
server_key_iv: "00eff6564817835b5040c02f16a8e0ba"
server_key_version: "encryptionkey@0"
I20260806 15:07:24.218600 1654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-1/wal/instance:
uuid: "48128ae3732b492197c9dda010961500"
format_stamp: "Formatted at 2026-08-06 15:07:24 on dist-test-slave-tc2s"
server_key: "c0cb0d34805b6b981e8117627758ceb8"
server_key_iv: "00eff6564817835b5040c02f16a8e0ba"
server_key_version: "encryptionkey@0"
I20260806 15:07:24.219763 1654 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:07:24.220378 1674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:24.220512 1654 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:07:24.220597 1654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-1/wal
uuid: "48128ae3732b492197c9dda010961500"
format_stamp: "Formatted at 2026-08-06 15:07:24 on dist-test-slave-tc2s"
server_key: "c0cb0d34805b6b981e8117627758ceb8"
server_key_iv: "00eff6564817835b5040c02f16a8e0ba"
server_key_version: "encryptionkey@0"
I20260806 15:07:24.220654 1654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:07:24.227013 1643 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.236.125" port: 37281 } attempt: 2
I20260806 15:07:24.227366 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.227238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:37281 (local address 127.0.0.1:33162)
0806 15:07:24.227294 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:24.227332 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.125:37281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 15:07:24.227669 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.227599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:37281 (local address 127.0.0.1:33178)
0806 15:07:24.227630 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:24.227654 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.125:37281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260806 15:07:24.227753 1643 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.236.125:37281: Network error: Client connection negotiation failed: client connection to 127.0.236.125:37281: connect: Connection refused (error 111)
I20260806 15:07:24.231369 1654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:07:24.233283 1654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:07:24.233407 1654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:07:24.237016 1654 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.125:37281
I20260806 15:07:24.237057 1749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.125:37281 every 8 connection(s)
I20260806 15:07:24.237331 1654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-1/data/info.pb
I20260806 15:07:24.238345 1751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:07:24.239725 1751 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41749 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:07:24.240002 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 1654
I20260806 15:07:24.240098 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-1/wal/instance
I20260806 15:07:24.240299 945 external_mini_cluster.cc:1428] Setting key eae1271eaa7141b234ab3d485d72e492
Aug 06 15:07:24 dist-test-slave-tc2s krb5kdc[1520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.125@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.236.124@KRBTEST.COM; defaulting to no policy
I20260806 15:07:24.248296 1648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.240415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39578 (local address 127.0.236.126:41749)
0806 15:07:24.240463 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.240465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.240680 (+ 215us) server_negotiation.cc:407] Connection header received
0806 15:07:24.241164 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.241165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.241177 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.241227 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.241803 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.242323 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.243272 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.243477 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.243532 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:24.245666 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:24.245667 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:24.245667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:24.245674 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:24.246963 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.247413 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.247414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.247414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.247463 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.247714 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.247714 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.247715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.247812 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:24.247866 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:24.248163 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 15:07:24.248233 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 15:07:24.248512 1753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.240408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:41749 (local address 127.0.0.1:39578)
0806 15:07:24.240563 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:24.240596 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 15:07:24.241068 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:07:24.241291 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:07:24.241294 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:07:24.241536 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:07:24.241714 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:24.241718 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.242407 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.242408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:24.243184 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:24.243186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.243559 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.243561 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:24.243682 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:24.244114 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:07:24.244122 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:07:24.245571 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:07:24.247044 (+ 1473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:24.247051 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:24.247054 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:24.247337 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:24.247519 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:24.247520 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:24.247520 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:24.247652 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:24.247867 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:07:24.247872 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:07:24.248027 (+ 155us) client_negotiation.cc:769] Sending connection context
0806 15:07:24.248142 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 15:07:24.248301 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
I20260806 15:07:24.248966 1751 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.125" port: 37281 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.124@KRBTEST.COM" created.
Aug 06 15:07:24 dist-test-slave-tc2s krb5kdc[1520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.125@KRBTEST.COM for kudu/127.0.236.125@KRBTEST.COM
I20260806 15:07:24.255476 1753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.249192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:37281 (local address 127.0.0.1:33184)
0806 15:07:24.249236 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:24.249247 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:07:24.249288 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:07:24.249512 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:07:24.249513 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:07:24.249616 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:07:24.249660 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:24.249661 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.250462 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.250464 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:24.251014 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:24.251017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.251455 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.251457 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:24.251518 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:24.251848 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:07:24.251851 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:07:24.253035 (+ 1184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:07:24.254556 (+ 1521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:24.254556 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:24.254557 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:24.254708 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:24.254881 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:24.254882 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:24.254882 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:24.254934 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:24.255173 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:07:24.255174 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:07:24.255211 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 15:07:24.255273 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 15:07:24.255366 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 15:07:24.255563 1756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.249269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33184 (local address 127.0.236.125:37281)
0806 15:07:24.249397 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.249400 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.249414 (+ 14us) server_negotiation.cc:407] Connection header received
0806 15:07:24.249431 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.249433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.249442 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.249498 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.249715 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.250392 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.251087 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.251385 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.251442 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:24.253105 (+ 1663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:24.253113 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:24.253113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:24.253119 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:24.254476 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.254774 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.254777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.254777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.254829 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.254984 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.254985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.254985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.255115 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:24.255182 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:24.255307 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 15:07:24.255436 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":107,"thread_start_us":82,"threads_started":1}
I20260806 15:07:24.256026 1751 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:07:24.256319 1753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.256226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:41821 (local address 127.0.0.1:55858)
0806 15:07:24.256263 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:24.256300 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:41821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 15:07:24.256414 1675 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.236.124:41821: connect: Connection refused (error 111)
I20260806 15:07:24.256778 1753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.256695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:41821 (local address 127.0.0.1:55860)
0806 15:07:24.256731 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:24.256754 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:41821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 15:07:24.256866 1751 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.236.124:41821: Network error: Client connection negotiation failed: client connection to 127.0.236.124:41821: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.124.keytab.
Entry for principal kudu/127.0.236.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.124.keytab.
I20260806 15:07:24.263854 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.124:41821
--webserver_interface=127.0.236.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.124.keytab
--principal=kudu/127.0.236.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:44645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:41749,127.0.236.125:37281,127.0.236.124:41821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:07:24.306056 1751 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 } attempt: 1
I20260806 15:07:24.306531 1753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.306402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:41821 (local address 127.0.0.1:55864)
0806 15:07:24.306466 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:24.306503 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:41821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 15:07:24.306847 1753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.306768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:41821 (local address 127.0.0.1:55876)
0806 15:07:24.306806 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:24.306829 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:41821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 15:07:24.306924 1751 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.236.124:41821: Network error: Client connection negotiation failed: client connection to 127.0.236.124:41821: connect: Connection refused (error 111)
I20260806 15:07:24.318902 1643 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.236.125" port: 37281 } attempt: 3
Aug 06 15:07:24 dist-test-slave-tc2s krb5kdc[1520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for kudu/127.0.236.125@KRBTEST.COM
I20260806 15:07:24.326021 1756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.319266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33192 (local address 127.0.236.125:37281)
0806 15:07:24.319333 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.319335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.319347 (+ 12us) server_negotiation.cc:407] Connection header received
0806 15:07:24.319379 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.319380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.319391 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.319451 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.319740 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.320523 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.321179 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.321530 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.321601 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:24.323612 (+ 2011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:24.323613 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:24.323614 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:24.323620 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:24.324966 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.325339 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.325339 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.325340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.325386 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.325558 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.325558 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.325558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.325654 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:24.325698 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:24.325859 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 15:07:24.325955 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 15:07:24.326069 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.319214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:37281 (local address 127.0.0.1:33192)
0806 15:07:24.319272 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:24.319284 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:07:24.319340 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:07:24.319457 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:07:24.319457 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:07:24.319607 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:07:24.319674 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:24.319676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.320595 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.320596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:24.321110 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:24.321111 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.321594 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.321595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:24.321650 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:24.322093 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:07:24.322097 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:07:24.323530 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:07:24.325050 (+ 1520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:24.325051 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:24.325052 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:24.325271 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:24.325434 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:24.325434 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:24.325434 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:24.325505 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:24.325706 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:07:24.325707 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:07:24.325761 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 15:07:24.325844 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 15:07:24.325970 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 15:07:24.326449 1643 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:07:24.326786 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.326676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:41821 (local address 127.0.0.1:55884)
0806 15:07:24.326723 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:24.326761 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:41821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 15:07:24.327191 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.327101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:41821 (local address 127.0.0.1:55890)
0806 15:07:24.327143 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:24.327170 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:41821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 15:07:24.327279 1643 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.236.124:41821: Network error: Client connection negotiation failed: client connection to 127.0.236.124:41821: connect: Connection refused (error 111)
W20260806 15:07:24.344533 1759 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 15:07:24.344722 1759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:07:24.344738 1759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:07:24.344764 1759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:07:24.345814 1759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:07:24.345846 1759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:07:24.345858 1759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:07:24.345870 1759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:07:24.345885 1759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:07:24.347354 1759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:44645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:41749,127.0.236.125:37281,127.0.236.124:41821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/krb5kdc/kudu_127.0.236.124.keytab
--principal=kudu/127.0.236.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.124:41821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.236.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:07:24.347566 1759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:07:24.347785 1759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:07:24.352190 1767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:24.352221 1770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:24.352206 1765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:24.352353 1759 server_base.cc:1034] running on GCE node
I20260806 15:07:24.352660 1759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:07:24.352931 1759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:07:24.354063 1759 hybrid_clock.cc:648] HybridClock initialized: now 1786028844354050 us; error 26 us; skew 500 ppm
Aug 06 15:07:24 dist-test-slave-tc2s krb5kdc[1520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:07:24.356302 1759 init.cc:377] Logged in from keytab as kudu/127.0.236.124@KRBTEST.COM (short username kudu)
I20260806 15:07:24.356710 1759 webserver.cc:460] Webserver started at http://127.0.236.124:34331/ using document root <none> and password file <none>
I20260806 15:07:24.356884 1759 fs_manager.cc:359] Metadata directory not provided
I20260806 15:07:24.356928 1759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:07:24.357018 1759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:07:24.358002 1759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-2/data/instance:
uuid: "8c1f9a7377644afea8f08fddc8451c8a"
format_stamp: "Formatted at 2026-08-06 15:07:24 on dist-test-slave-tc2s"
server_key: "8dafa36d431788c843b16f8e9b7039e4"
server_key_iv: "401cdc01f8a583f11ecf1df0523ef601"
server_key_version: "encryptionkey@0"
I20260806 15:07:24.358285 1759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-2/wal/instance:
uuid: "8c1f9a7377644afea8f08fddc8451c8a"
format_stamp: "Formatted at 2026-08-06 15:07:24 on dist-test-slave-tc2s"
server_key: "8dafa36d431788c843b16f8e9b7039e4"
server_key_iv: "401cdc01f8a583f11ecf1df0523ef601"
server_key_version: "encryptionkey@0"
I20260806 15:07:24.358433 1643 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 } attempt: 1
I20260806 15:07:24.358767 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.358656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:41821 (local address 127.0.0.1:55898)
0806 15:07:24.358710 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:24.358738 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:41821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 15:07:24.359098 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.359021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:41821 (local address 127.0.0.1:55904)
0806 15:07:24.359057 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:24.359081 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:41821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 15:07:24.359195 1643 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.236.124:41821: Network error: Client connection negotiation failed: client connection to 127.0.236.124:41821: connect: Connection refused (error 111)
I20260806 15:07:24.359777 1759 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 15:07:24.360389 1780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:24.360546 1759 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:07:24.360626 1759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-2/wal
uuid: "8c1f9a7377644afea8f08fddc8451c8a"
format_stamp: "Formatted at 2026-08-06 15:07:24 on dist-test-slave-tc2s"
server_key: "8dafa36d431788c843b16f8e9b7039e4"
server_key_iv: "401cdc01f8a583f11ecf1df0523ef601"
server_key_version: "encryptionkey@0"
I20260806 15:07:24.360678 1759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:07:24.375049 1751 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 } attempt: 2
I20260806 15:07:24.375413 1753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.375271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:41821 (local address 127.0.0.1:55908)
0806 15:07:24.375335 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:24.375372 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:41821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 15:07:24.375751 1753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.375661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:41821 (local address 127.0.0.1:55924)
0806 15:07:24.375702 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:24.375729 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:41821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 15:07:24.375845 1751 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.236.124:41821: Network error: Client connection negotiation failed: client connection to 127.0.236.124:41821: connect: Connection refused (error 111)
I20260806 15:07:24.376325 1759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:07:24.378463 1759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:07:24.378582 1759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:07:24.382196 1759 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.124:41821
I20260806 15:07:24.382225 1855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.124:41821 every 8 connection(s)
I20260806 15:07:24.382521 1759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-2/data/info.pb
I20260806 15:07:24.383466 1856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:07:24.384867 1856 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41749 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:07:24.388370 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 1759
I20260806 15:07:24.388459 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028580791223-945-0/minicluster-data/master-2/wal/instance
I20260806 15:07:24.388635 945 external_mini_cluster.cc:1428] Setting key a7858947693da2e2699b45a4b15a13ce
Aug 06 15:07:24 dist-test-slave-tc2s krb5kdc[1520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.124@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:07:24.393617 1648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.385509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39580 (local address 127.0.236.126:41749)
0806 15:07:24.385558 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.385560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.385736 (+ 176us) server_negotiation.cc:407] Connection header received
0806 15:07:24.386239 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.386241 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.386254 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.386305 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.386884 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.387352 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.388341 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.388565 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.388621 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:24.390866 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:24.390867 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:24.390868 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:24.390875 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:24.392216 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.392673 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.392674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.392675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.392745 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.393005 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.393006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.393006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.393115 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:24.393164 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:24.393495 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 15:07:24.393557 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 15:07:24.393841 1858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.385479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:41749 (local address 127.0.0.1:39580)
0806 15:07:24.385625 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:24.385659 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 15:07:24.386146 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:07:24.386370 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:07:24.386374 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:07:24.386613 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:07:24.386792 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:24.386795 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.387431 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.387432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:24.388236 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:24.388239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.388658 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.388660 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:24.388829 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:24.389284 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:07:24.389294 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:07:24.390785 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:07:24.392291 (+ 1506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:24.392298 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:24.392299 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:24.392601 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:24.392803 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:24.392803 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:24.392804 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:24.392941 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:24.393173 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:07:24.393180 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:07:24.393363 (+ 183us) client_negotiation.cc:769] Sending connection context
0806 15:07:24.393484 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 15:07:24.393628 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
Aug 06 15:07:24 dist-test-slave-tc2s krb5kdc[1520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.124@KRBTEST.COM
I20260806 15:07:24.394335 1856 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.125" port: 37281 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:07:24.397552 1862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.389002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55928 (local address 127.0.236.124:41821)
0806 15:07:24.389187 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.389192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.389285 (+ 93us) server_negotiation.cc:407] Connection header received
0806 15:07:24.389355 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.389357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.389375 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.389432 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.390196 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.390960 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.391867 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.392319 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.392372 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:24.394833 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:24.394841 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:24.394841 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:24.394847 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:24.396392 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.396748 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.396751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.396751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.396805 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.397002 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.397003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.397004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.397150 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:24.397306 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:24.397378 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 15:07:24.397457 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":103,"threads_started":1}
Aug 06 15:07:24 dist-test-slave-tc2s krb5kdc[1520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.124@KRBTEST.COM for kudu/127.0.236.125@KRBTEST.COM
I20260806 15:07:24.400928 1858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.394572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:37281 (local address 127.0.0.1:33206)
0806 15:07:24.394666 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:24.394675 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:07:24.394714 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:07:24.394856 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:07:24.394858 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:07:24.395015 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:07:24.395066 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:24.395068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.395781 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.395782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:24.396505 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:24.396507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.396891 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.396892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:24.396953 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:24.397249 (+ 296us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:07:24.397253 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:07:24.398542 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:07:24.400078 (+ 1536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:24.400079 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:24.400079 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:24.400228 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:24.400410 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:24.400411 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:24.400411 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:24.400471 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:24.400674 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:07:24.400675 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:07:24.400712 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 15:07:24.400760 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 15:07:24.400842 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 15:07:24.400933 1756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.394625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33206 (local address 127.0.236.125:37281)
0806 15:07:24.394675 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.394677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.394708 (+ 31us) server_negotiation.cc:407] Connection header received
0806 15:07:24.394754 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.394755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.394764 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.394832 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.395130 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.395712 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.396568 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.396828 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.396884 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:24.398602 (+ 1718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:24.398603 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:24.398604 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:24.398612 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:24.400004 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.400289 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.400289 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.400290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.400361 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.400524 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.400525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.400525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.400618 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:24.400674 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:24.400802 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 15:07:24.400873 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 15:07:24.401413 1856 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 15:07:24 dist-test-slave-tc2s krb5kdc[1520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
Aug 06 15:07:24 dist-test-slave-tc2s krb5kdc[1520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.124@KRBTEST.COM for kudu/127.0.236.124@KRBTEST.COM
I20260806 15:07:24.405375 1648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.398373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39590 (local address 127.0.236.126:41749)
0806 15:07:24.398424 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.398426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.398432 (+ 6us) server_negotiation.cc:407] Connection header received
0806 15:07:24.398455 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.398456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.398463 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.398504 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.399017 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.399444 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.400299 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.400570 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.400620 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:24.402892 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:24.402893 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:24.402894 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:24.402899 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:24.404234 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.404590 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.404591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.404591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.404644 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.404806 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.404807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.404807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.404935 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:24.405102 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:24.405139 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 15:07:24.405207 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 15:07:24.407866 1858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.401632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:41821 (local address 127.0.0.1:55932)
0806 15:07:24.401723 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:24.401732 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:07:24.401782 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:07:24.401920 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:07:24.401920 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:07:24.402034 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:07:24.402061 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:24.402062 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.403088 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.403088 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:24.403756 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:24.403759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.404090 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.404091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:24.404148 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:24.404467 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:07:24.404471 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:07:24.405512 (+ 1041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:07:24.407021 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:24.407021 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:24.407022 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:24.407217 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:24.407396 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:24.407396 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:24.407396 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:24.407440 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:24.407655 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:07:24.407656 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:07:24.407688 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 15:07:24.407728 (+ 40us) client_negotiation.cc:240] Negotiation successful
0806 15:07:24.407792 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 15:07:24.407965 1862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.401684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55932 (local address 127.0.236.124:41821)
0806 15:07:24.401718 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.401720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.401767 (+ 47us) server_negotiation.cc:407] Connection header received
0806 15:07:24.401836 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.401836 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.401844 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.401884 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.402129 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.402818 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.403827 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.404021 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.404071 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:24.405589 (+ 1518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:24.405590 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:24.405590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:24.405596 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:24.406947 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.407276 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.407277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.407277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.407337 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.407489 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.407490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.407490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.407603 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:24.407655 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:24.407750 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 15:07:24.407849 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":18}
I20260806 15:07:24.409224 1856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c1f9a7377644afea8f08fddc8451c8a: Bootstrap starting.
Aug 06 15:07:24 dist-test-slave-tc2s krb5kdc[1520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.125@KRBTEST.COM
I20260806 15:07:24.410173 1856 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c1f9a7377644afea8f08fddc8451c8a: Neither blocks nor log segments found. Creating new log.
I20260806 15:07:24.410429 1856 log.cc:824] T 00000000000000000000000000000000 P 8c1f9a7377644afea8f08fddc8451c8a: Log is configured to *not* fsync() on all Append() calls
I20260806 15:07:24.411202 1856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c1f9a7377644afea8f08fddc8451c8a: No bootstrap required, opened a new log
I20260806 15:07:24.412590 1756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.405942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33214 (local address 127.0.236.125:37281)
0806 15:07:24.405985 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.405987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.405994 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:07:24.406013 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.406013 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.406021 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.406065 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.406594 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.407040 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.407857 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.408132 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.408197 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:24.410367 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:24.410368 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:24.410368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:24.410374 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:24.411660 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.411955 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.411956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.411956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.412007 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.412163 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.412164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.412165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.412266 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:24.412404 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:24.412439 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 15:07:24.412506 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 15:07:24.412562 1856 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c1f9a7377644afea8f08fddc8451c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41749 } } peers { permanent_uuid: "48128ae3732b492197c9dda010961500" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 37281 } } peers { permanent_uuid: "8c1f9a7377644afea8f08fddc8451c8a" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 } }
I20260806 15:07:24.412865 1856 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c1f9a7377644afea8f08fddc8451c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c1f9a7377644afea8f08fddc8451c8a, State: Initialized, Role: FOLLOWER
I20260806 15:07:24.412966 1856 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c1f9a7377644afea8f08fddc8451c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41749 } } peers { permanent_uuid: "48128ae3732b492197c9dda010961500" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 37281 } } peers { permanent_uuid: "8c1f9a7377644afea8f08fddc8451c8a" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 } }
I20260806 15:07:24.413295 1856 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c1f9a7377644afea8f08fddc8451c8a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:07:24.414284 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:07:24.414681 1865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c1f9a7377644afea8f08fddc8451c8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41749 } } peers { permanent_uuid: "48128ae3732b492197c9dda010961500" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 37281 } } peers { permanent_uuid: "8c1f9a7377644afea8f08fddc8451c8a" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 } } }
W20260806 15:07:24.415091 1880 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8c1f9a7377644afea8f08fddc8451c8a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 15:07:24.415156 1865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c1f9a7377644afea8f08fddc8451c8a [sys.catalog]: This master's current role is: FOLLOWER
W20260806 15:07:24.415165 1880 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:07:24.415360 1600 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ae6eeacae6e0467da80ce2b75d92fc42"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39590
I20260806 15:07:24.415683 1759 master_runner.cc:186] Error getting master registration for 127.0.236.126:41749: OK, instance_id { permanent_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" instance_seqno: 1786028844094154 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 15:07:24.417752 1759 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 15:07:24.434342 1643 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 } attempt: 2
Aug 06 15:07:24 dist-test-slave-tc2s krb5kdc[1520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for kudu/127.0.236.124@KRBTEST.COM
I20260806 15:07:24.441196 1862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.434683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55948 (local address 127.0.236.124:41821)
0806 15:07:24.434755 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.434757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.435238 (+ 481us) server_negotiation.cc:407] Connection header received
0806 15:07:24.435262 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.435262 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.435272 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.435314 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.435697 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.436162 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.436950 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.437137 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.437192 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:24.438900 (+ 1708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:24.438902 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:24.438902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:24.438909 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:24.440120 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.440436 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.440437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.440438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.440493 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.440654 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.440655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.440656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.440784 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:24.440842 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:24.440980 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 15:07:24.441046 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 15:07:24.441136 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.434617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:41821 (local address 127.0.0.1:55948)
0806 15:07:24.435111 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:24.435124 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:07:24.435183 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:07:24.435361 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:07:24.435362 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:07:24.435553 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:07:24.435623 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:24.435625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.436233 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.436234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:24.436874 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:24.436877 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.437195 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.437196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:24.437253 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:24.437647 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:07:24.437652 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:07:24.438826 (+ 1174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:07:24.440194 (+ 1368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:24.440195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:24.440196 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:24.440375 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:24.440546 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:24.440548 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:24.440548 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:24.440605 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:24.440842 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:07:24.440843 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:07:24.440889 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 15:07:24.440948 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 15:07:24.441042 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469}
I20260806 15:07:24.442823 1643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42: Bootstrap starting.
I20260806 15:07:24.443552 1643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42: Neither blocks nor log segments found. Creating new log.
I20260806 15:07:24.443851 1643 log.cc:824] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42: Log is configured to *not* fsync() on all Append() calls
I20260806 15:07:24.444689 1643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42: No bootstrap required, opened a new log
I20260806 15:07:24.446046 1643 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41749 } } peers { permanent_uuid: "48128ae3732b492197c9dda010961500" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 37281 } } peers { permanent_uuid: "8c1f9a7377644afea8f08fddc8451c8a" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 } }
I20260806 15:07:24.446182 1643 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae6eeacae6e0467da80ce2b75d92fc42, State: Initialized, Role: FOLLOWER
I20260806 15:07:24.446311 1643 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41749 } } peers { permanent_uuid: "48128ae3732b492197c9dda010961500" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 37281 } } peers { permanent_uuid: "8c1f9a7377644afea8f08fddc8451c8a" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 } }
I20260806 15:07:24.446666 1643 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:07:24.446630 1883 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41749 } } peers { permanent_uuid: "48128ae3732b492197c9dda010961500" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 37281 } } peers { permanent_uuid: "8c1f9a7377644afea8f08fddc8451c8a" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 } } }
I20260806 15:07:24.446735 1883 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:07:24.446807 1600 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 15:07:24.446868 1600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:07:24.447538 1600 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41749 } } peers { permanent_uuid: "48128ae3732b492197c9dda010961500" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 37281 } } peers { permanent_uuid: "8c1f9a7377644afea8f08fddc8451c8a" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 } }
I20260806 15:07:24.448295 1818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8c1f9a7377644afea8f08fddc8451c8a"
I20260806 15:07:24.448421 1818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c1f9a7377644afea8f08fddc8451c8a [term 0 FOLLOWER]: Advancing to term 1
W20260806 15:07:24.448817 1896 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:07:24.448860 1896 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:07:24.449301 1818 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8c1f9a7377644afea8f08fddc8451c8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae6eeacae6e0467da80ce2b75d92fc42 in term 1.
I20260806 15:07:24.449570 1566 leader_election.cc:302] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c1f9a7377644afea8f08fddc8451c8a, ae6eeacae6e0467da80ce2b75d92fc42; no voters:
I20260806 15:07:24.449700 1883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:07:24.449812 1883 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42 [term 1 LEADER]: Becoming Leader. State: Replica: ae6eeacae6e0467da80ce2b75d92fc42, State: Running, Role: LEADER
I20260806 15:07:24.449882 1883 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41749 } } peers { permanent_uuid: "48128ae3732b492197c9dda010961500" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 37281 } } peers { permanent_uuid: "8c1f9a7377644afea8f08fddc8451c8a" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 } }
I20260806 15:07:24.450305 1883 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae6eeacae6e0467da80ce2b75d92fc42. Latest consensus state: current_term: 1 leader_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41749 } } peers { permanent_uuid: "48128ae3732b492197c9dda010961500" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 37281 } } peers { permanent_uuid: "8c1f9a7377644afea8f08fddc8451c8a" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 } } }
I20260806 15:07:24.450395 1883 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42 [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:24.450682 1600 leader_election.cc:288] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42 [CANDIDATE]: Term 1 election: Requested vote from peers 48128ae3732b492197c9dda010961500 (127.0.236.125:37281), 8c1f9a7377644afea8f08fddc8451c8a (127.0.236.124:41821)
I20260806 15:07:24.449133 1544 master_runner.cc:186] Error getting master registration for 127.0.236.125:37281: OK, instance_id { permanent_uuid: "48128ae3732b492197c9dda010961500" instance_seqno: 1786028844236042 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 15:07:24.451332 1544 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 15:07:24.454003 1903 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:07:24.454233 1903 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:07:24.459692 1756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.452105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33230 (local address 127.0.236.125:37281)
0806 15:07:24.452145 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.452147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.452358 (+ 211us) server_negotiation.cc:407] Connection header received
0806 15:07:24.452463 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.452464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.452474 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.452503 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.453265 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.454032 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.455056 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.455378 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.455418 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:24.456893 (+ 1475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:24.456894 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:24.456895 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:24.456902 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:24.458485 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.458866 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.458868 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.458868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.458922 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.459108 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.459109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.459109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.459228 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:24.459304 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:24.459521 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 15:07:24.459597 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:07:24.462294 1648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.451880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39598 (local address 127.0.236.126:41749)
0806 15:07:24.451923 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.451925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.452137 (+ 212us) server_negotiation.cc:407] Connection header received
0806 15:07:24.452277 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.452278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.452299 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.452328 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.453572 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.454250 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.455729 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.456022 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.456062 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:24.458710 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:24.458711 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:24.458712 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:24.458719 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:24.460604 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.461156 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.461157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.461158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.461215 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.461532 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.461533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.461533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.461676 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:24.461760 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:24.462136 (+ 376us) server_negotiation.cc:299] Negotiation successful
0806 15:07:24.462209 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 15:07:24.463284 1818 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8c1f9a7377644afea8f08fddc8451c8a [term 1 FOLLOWER]: Refusing update from remote peer ae6eeacae6e0467da80ce2b75d92fc42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:07:24.463721 1900 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c1f9a7377644afea8f08fddc8451c8a" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:07:24.465106 1865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c1f9a7377644afea8f08fddc8451c8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae6eeacae6e0467da80ce2b75d92fc42. Latest consensus state: current_term: 1 leader_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41749 } } peers { permanent_uuid: "48128ae3732b492197c9dda010961500" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 37281 } } peers { permanent_uuid: "8c1f9a7377644afea8f08fddc8451c8a" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 } } }
I20260806 15:07:24.465195 1865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c1f9a7377644afea8f08fddc8451c8a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:07:24.467015 1751 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 } attempt: 3
I20260806 15:07:24.467490 1900 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41749 } } peers { permanent_uuid: "48128ae3732b492197c9dda010961500" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 37281 } } peers { permanent_uuid: "8c1f9a7377644afea8f08fddc8451c8a" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 } } }
I20260806 15:07:24.467577 1900 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42 [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:24.467895 1865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c1f9a7377644afea8f08fddc8451c8a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41749 } } peers { permanent_uuid: "48128ae3732b492197c9dda010961500" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 37281 } } peers { permanent_uuid: "8c1f9a7377644afea8f08fddc8451c8a" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 } } }
I20260806 15:07:24.467976 1865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c1f9a7377644afea8f08fddc8451c8a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:07:24.468577 1862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.459132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55962 (local address 127.0.236.124:41821)
0806 15:07:24.459166 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.459168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.460512 (+ 1344us) server_negotiation.cc:407] Connection header received
0806 15:07:24.460598 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.460599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.460608 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.460634 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.461281 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.462017 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.462949 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.463199 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.463237 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:24.464576 (+ 1339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:24.464577 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:24.464578 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:24.464584 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:24.466204 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.466572 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.466573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.466573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.466645 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.466832 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.466833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.466834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.466953 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:24.467034 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:24.468424 (+ 1390us) server_negotiation.cc:299] Negotiation successful
0806 15:07:24.468493 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Aug 06 15:07:24 dist-test-slave-tc2s krb5kdc[1520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.125@KRBTEST.COM for kudu/127.0.236.124@KRBTEST.COM
I20260806 15:07:24.474242 1903 catalog_manager.cc:1347] Generated new cluster ID: 22ba1a403ebe45b2ac9a0b044212e0b4
I20260806 15:07:24.474287 1903 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:07:24.476382 1753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.467283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:41821 (local address 127.0.0.1:55964)
0806 15:07:24.467358 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:24.467379 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 15:07:24.467415 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:07:24.468757 (+ 1342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:07:24.468758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:07:24.468902 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:07:24.468940 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:24.468942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.469864 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.469865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:24.470676 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:24.470679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.471137 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.471138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:24.471205 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:24.471597 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:07:24.471602 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:07:24.473744 (+ 2142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:07:24.475449 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:24.475450 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:24.475450 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:24.475642 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:24.475821 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:24.475821 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:24.475835 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:24.475885 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:24.476125 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:07:24.476125 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:07:24.476163 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 15:07:24.476216 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 15:07:24.476295 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 15:07:24.476406 1862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:24.467517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55964 (local address 127.0.236.124:41821)
0806 15:07:24.468639 (+ 1122us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.468640 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.468647 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:07:24.468662 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.468663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.468671 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.468719 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.469104 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.469791 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.470757 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:24.471045 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:24.471118 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:24.473826 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:24.473827 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:24.473827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:24.473834 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:24.475353 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.475704 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.475705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.475705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.475761 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:24.475934 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:24.475935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:24.475935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:24.476054 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:24.476119 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:24.476246 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 15:07:24.476313 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1109,"thread_start_us":71,"threads_started":1}
I20260806 15:07:24.478263 1751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48128ae3732b492197c9dda010961500: Bootstrap starting.
I20260806 15:07:24.478960 1751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48128ae3732b492197c9dda010961500: Neither blocks nor log segments found. Creating new log.
I20260806 15:07:24.479249 1751 log.cc:824] T 00000000000000000000000000000000 P 48128ae3732b492197c9dda010961500: Log is configured to *not* fsync() on all Append() calls
I20260806 15:07:24.482496 1751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48128ae3732b492197c9dda010961500: No bootstrap required, opened a new log
I20260806 15:07:24.484477 1751 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48128ae3732b492197c9dda010961500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41749 } } peers { permanent_uuid: "48128ae3732b492197c9dda010961500" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 37281 } } peers { permanent_uuid: "8c1f9a7377644afea8f08fddc8451c8a" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 } }
I20260806 15:07:24.484637 1751 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48128ae3732b492197c9dda010961500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48128ae3732b492197c9dda010961500, State: Initialized, Role: FOLLOWER
I20260806 15:07:24.484719 1751 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48128ae3732b492197c9dda010961500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41749 } } peers { permanent_uuid: "48128ae3732b492197c9dda010961500" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 37281 } } peers { permanent_uuid: "8c1f9a7377644afea8f08fddc8451c8a" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 } }
I20260806 15:07:24.485049 1751 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48128ae3732b492197c9dda010961500 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:07:24.485108 1713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "48128ae3732b492197c9dda010961500"
I20260806 15:07:24.485215 1713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48128ae3732b492197c9dda010961500 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:07:24.486101 1713 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 48128ae3732b492197c9dda010961500 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae6eeacae6e0467da80ce2b75d92fc42 in term 1.
I20260806 15:07:24.486338 1712 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 48128ae3732b492197c9dda010961500 [term 1 FOLLOWER]: Refusing update from remote peer ae6eeacae6e0467da80ce2b75d92fc42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260806 15:07:24.486385 1684 tablet.cc:2369] T 00000000000000000000000000000000 P 48128ae3732b492197c9dda010961500: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:07:24.487236 1883 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48128ae3732b492197c9dda010961500" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 37281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:07:24.489216 1941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48128ae3732b492197c9dda010961500 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41749 } } peers { permanent_uuid: "48128ae3732b492197c9dda010961500" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 37281 } } peers { permanent_uuid: "8c1f9a7377644afea8f08fddc8451c8a" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 } } }
I20260806 15:07:24.489310 1941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48128ae3732b492197c9dda010961500 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:07:24.489360 1941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48128ae3732b492197c9dda010961500 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41749 } } peers { permanent_uuid: "48128ae3732b492197c9dda010961500" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 37281 } } peers { permanent_uuid: "8c1f9a7377644afea8f08fddc8451c8a" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 } } }
I20260806 15:07:24.489409 1941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48128ae3732b492197c9dda010961500 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:07:24.489799 1946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48128ae3732b492197c9dda010961500 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae6eeacae6e0467da80ce2b75d92fc42. Latest consensus state: current_term: 1 leader_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41749 } } peers { permanent_uuid: "48128ae3732b492197c9dda010961500" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 37281 } } peers { permanent_uuid: "8c1f9a7377644afea8f08fddc8451c8a" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 } } }
I20260806 15:07:24.489871 1946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48128ae3732b492197c9dda010961500 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:07:24.494215 1940 catalog_manager.cc:1259] Loaded cluster ID: 22ba1a403ebe45b2ac9a0b044212e0b4
I20260806 15:07:24.494266 1940 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 48128ae3732b492197c9dda010961500: loading cluster ID for follower catalog manager: success
W20260806 15:07:24.494355 1940 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 48128ae3732b492197c9dda010961500: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260806 15:07:24.494392 1940 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:07:24.496253 1903 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:07:24.496919 1903 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:07:24.498625 1900 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae6eeacae6e0467da80ce2b75d92fc42" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 41749 } } peers { permanent_uuid: "48128ae3732b492197c9dda010961500" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 37281 } } peers { permanent_uuid: "8c1f9a7377644afea8f08fddc8451c8a" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 41821 } } }
I20260806 15:07:24.498716 1900 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42 [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:24.514987 1903 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae6eeacae6e0467da80ce2b75d92fc42: Generated new TSK 0
I20260806 15:07:24.515183 1903 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 15:07:24.541728 1648 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:07:24.540454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39602 (local address 127.0.236.126:41749)
0806 15:07:24.540499 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.540501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.541093 (+ 592us) server_negotiation.cc:407] Connection header received
0806 15:07:24.541194 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.541195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.541206 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.541236 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.541660 (+ 424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39602: BlockingRecv error: recv got EOF from 127.0.0.1:39602 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 15:07:24.542737 1756 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:07:24.540800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33246 (local address 127.0.236.125:37281)
0806 15:07:24.540851 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.540853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.542203 (+ 1350us) server_negotiation.cc:407] Connection header received
0806 15:07:24.542340 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.542341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.542351 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.542380 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.542676 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33246: BlockingRecv error: recv got EOF from 127.0.0.1:33246 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260806 15:07:24.543507 1862 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:07:24.542087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55980 (local address 127.0.236.124:41821)
0806 15:07:24.542126 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.542128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.542941 (+ 813us) server_negotiation.cc:407] Connection header received
0806 15:07:24.543099 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.543100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.543109 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.543136 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.543450 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55980: BlockingRecv error: recv got EOF from 127.0.0.1:55980 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 15:07:24.545627 1756 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:07:24.544948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33258 (local address 127.0.236.125:37281)
0806 15:07:24.544981 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.544983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.545159 (+ 176us) server_negotiation.cc:407] Connection header received
0806 15:07:24.545254 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.545255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.545271 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.545297 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.545584 (+ 287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33258: BlockingRecv error: recv got EOF from 127.0.0.1:33258 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 15:07:24.548601 1756 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:07:24.547840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33264 (local address 127.0.236.125:37281)
0806 15:07:24.547870 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.547872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.548140 (+ 268us) server_negotiation.cc:407] Connection header received
0806 15:07:24.548225 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.548226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.548234 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.548259 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.548560 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33264: BlockingRecv error: recv got EOF from 127.0.0.1:33264 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 15:07:24.552253 1756 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:07:24.551539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33270 (local address 127.0.236.125:37281)
0806 15:07:24.551572 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.551573 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.551789 (+ 216us) server_negotiation.cc:407] Connection header received
0806 15:07:24.551891 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.551892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.551902 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.551930 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.552214 (+ 284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33270: BlockingRecv error: recv got EOF from 127.0.0.1:33270 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 15:07:24.557955 1756 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:07:24.557224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33278 (local address 127.0.236.125:37281)
0806 15:07:24.557255 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.557257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.557477 (+ 220us) server_negotiation.cc:407] Connection header received
0806 15:07:24.557561 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.557562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.557570 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.557594 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.557908 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33278: BlockingRecv error: recv got EOF from 127.0.0.1:33278 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 15:07:24.567675 1756 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:07:24.566911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33284 (local address 127.0.236.125:37281)
0806 15:07:24.566947 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.566948 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.567193 (+ 245us) server_negotiation.cc:407] Connection header received
0806 15:07:24.567278 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.567279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.567289 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.567316 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.567630 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33284: BlockingRecv error: recv got EOF from 127.0.0.1:33284 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 15:07:24.585875 1756 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:07:24.584851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33290 (local address 127.0.236.125:37281)
0806 15:07:24.584890 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.584892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.585312 (+ 420us) server_negotiation.cc:407] Connection header received
0806 15:07:24.585414 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.585415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.585426 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.585456 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.585820 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33290: BlockingRecv error: recv got EOF from 127.0.0.1:33290 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 15:07:24.620092 1756 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:07:24.619291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33304 (local address 127.0.236.125:37281)
0806 15:07:24.619337 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.619339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.619501 (+ 162us) server_negotiation.cc:407] Connection header received
0806 15:07:24.619564 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.619566 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.619579 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.619645 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.620031 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33304: BlockingRecv error: recv got EOF from 127.0.0.1:33304 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 15:07:24.685819 1756 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:07:24.685111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33310 (local address 127.0.236.125:37281)
0806 15:07:24.685172 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.685174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.685255 (+ 81us) server_negotiation.cc:407] Connection header received
0806 15:07:24.685390 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.685391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.685404 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.685478 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.685770 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33310: BlockingRecv error: recv got EOF from 127.0.0.1:33310 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260806 15:07:24.815623 1756 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:07:24.814954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33316 (local address 127.0.236.125:37281)
0806 15:07:24.815028 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:24.815031 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:24.815097 (+ 66us) server_negotiation.cc:407] Connection header received
0806 15:07:24.815172 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:24.815173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:24.815184 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:24.815234 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:24.815559 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33316: BlockingRecv error: recv got EOF from 127.0.0.1:33316 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260806 15:07:25.073354 1756 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:07:25.072754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33328 (local address 127.0.236.125:37281)
0806 15:07:25.072823 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:25.072825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:25.072835 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:07:25.072884 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:25.072885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:25.072896 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:25.072957 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:25.073292 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33328: BlockingRecv error: recv got EOF from 127.0.0.1:33328 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 15:07:25.415758 1880 catalog_manager.cc:1259] Loaded cluster ID: 22ba1a403ebe45b2ac9a0b044212e0b4
I20260806 15:07:25.415822 1880 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8c1f9a7377644afea8f08fddc8451c8a: loading cluster ID for follower catalog manager: success
I20260806 15:07:25.416626 1880 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8c1f9a7377644afea8f08fddc8451c8a: acquiring CA information for follower catalog manager: success
I20260806 15:07:25.416903 1880 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8c1f9a7377644afea8f08fddc8451c8a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 15:07:25.495467 1940 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 48128ae3732b492197c9dda010961500: acquiring CA information for follower catalog manager: success
I20260806 15:07:25.495813 1940 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 48128ae3732b492197c9dda010961500: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 15:07:25.588796 2048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:25.586484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33340 (local address 127.0.236.125:37281)
0806 15:07:25.586615 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:25.586617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:25.586650 (+ 33us) server_negotiation.cc:407] Connection header received
0806 15:07:25.586725 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:25.586726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:25.586780 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:25.586858 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:07:25.586982 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:25.587478 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:25.588169 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:25.588352 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:25.588394 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:25.588517 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:07:25.588625 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:07:25.588677 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:25.588694 (+ 17us) server_negotiation.cc:299] Negotiation successful
0806 15:07:25.588710 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":61,"threads_started":1}
I20260806 15:07:25.592796 2055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:25.590577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55984 (local address 127.0.236.124:41821)
0806 15:07:25.590695 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:25.590697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:25.590717 (+ 20us) server_negotiation.cc:407] Connection header received
0806 15:07:25.590753 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:25.590753 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:25.590786 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:25.590840 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:07:25.590918 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:25.591374 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:25.592087 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:25.592313 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:25.592365 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:25.592533 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:07:25.592625 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:07:25.592674 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:25.592704 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 15:07:25.592721 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":66,"threads_started":1}
I20260806 15:07:25.596940 2055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:25.594382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55992 (local address 127.0.236.124:41821)
0806 15:07:25.594415 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:25.594417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:25.594532 (+ 115us) server_negotiation.cc:407] Connection header received
0806 15:07:25.594619 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:25.594620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:25.594629 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:25.594688 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:07:25.594783 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:25.595312 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:25.596212 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:25.596501 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:25.596569 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:25.596710 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:07:25.596741 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:07:25.596768 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:25.596857 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 15:07:25.596877 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 15:07:25.597172 2048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:25.594506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33356 (local address 127.0.236.125:37281)
0806 15:07:25.594540 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:25.594542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:25.594633 (+ 91us) server_negotiation.cc:407] Connection header received
0806 15:07:25.594715 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:25.594716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:25.594725 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:25.594751 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:07:25.594895 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:25.595568 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:25.596448 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:25.596694 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:25.596754 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:25.596915 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:07:25.596948 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:07:25.596978 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:25.597084 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 15:07:25.597106 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260806 15:07:25.597642 2065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:25.594364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39610 (local address 127.0.236.126:41749)
0806 15:07:25.594455 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:25.594457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:25.594614 (+ 157us) server_negotiation.cc:407] Connection header received
0806 15:07:25.594743 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:25.594744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:25.594793 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:25.594844 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:07:25.595042 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:25.595753 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:25.596629 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:25.596894 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:25.597041 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:25.597289 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:07:25.597398 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:07:25.597454 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:25.597536 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 15:07:25.597561 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70,"thread_start_us":45,"threads_started":1}
I20260806 15:07:25.599166 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 1544
I20260806 15:07:25.611606 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 1654
I20260806 15:07:25.618831 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 1759
2026-08-06T15:07:25Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2870 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:07:25 dist-test-slave-tc2s krb5kdc[2071](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:07:25 dist-test-slave-tc2s krb5kdc[2071](info): set up 2 sockets
Aug 06 15:07:25 dist-test-slave-tc2s krb5kdc[2071](info): commencing operation
krb5kdc: starting...
W20260806 15:07:26.735669 945 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/krb5kdc/test-user.keytab.
Aug 06 15:07:26 dist-test-slave-tc2s krb5kdc[2071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:07:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:07:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.236.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
Entry for principal kudu/127.0.236.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab.
I20260806 15:07:26.850543 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:46183
--webserver_interface=127.0.236.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:43053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:46183,127.0.236.125:40801,127.0.236.124:33191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:07:26.930994 2095 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 15:07:26.931241 2095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:07:26.931258 2095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:07:26.931278 2095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:07:26.932425 2095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:07:26.932463 2095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:07:26.932476 2095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:07:26.932488 2095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:07:26.932498 2095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:07:26.934067 2095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:43053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:46183,127.0.236.125:40801,127.0.236.124:33191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/krb5kdc/kudu_127.0.236.126.keytab
--principal=kudu/127.0.236.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.126:46183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.236.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:07:26.934293 2095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:07:26.934512 2095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:07:26.939267 2108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:26.939275 2106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:26.939504 2095 server_base.cc:1034] running on GCE node
W20260806 15:07:26.939275 2104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:26.939888 2095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:07:26.940189 2095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:07:26.941373 2095 hybrid_clock.cc:648] HybridClock initialized: now 1786028846941360 us; error 41 us; skew 500 ppm
Aug 06 15:07:26 dist-test-slave-tc2s krb5kdc[2071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028846, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:07:26.947348 2095 init.cc:377] Logged in from keytab as kudu/127.0.236.126@KRBTEST.COM (short username kudu)
I20260806 15:07:26.947809 2095 webserver.cc:460] Webserver started at http://127.0.236.126:45279/ using document root <none> and password file <none>
I20260806 15:07:26.948015 2095 fs_manager.cc:359] Metadata directory not provided
I20260806 15:07:26.948064 2095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:07:26.948189 2095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:07:26.949257 2095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-0/data/instance:
uuid: "a735dfc7faf44634a35995bea09a8201"
format_stamp: "Formatted at 2026-08-06 15:07:26 on dist-test-slave-tc2s"
server_key: "bbf26989465974ae9fee84d13b231f8c"
server_key_iv: "20669d509fbe2a8b16f4c1c01d3b3a26"
server_key_version: "encryptionkey@0"
I20260806 15:07:26.949604 2095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-0/wal/instance:
uuid: "a735dfc7faf44634a35995bea09a8201"
format_stamp: "Formatted at 2026-08-06 15:07:26 on dist-test-slave-tc2s"
server_key: "bbf26989465974ae9fee84d13b231f8c"
server_key_iv: "20669d509fbe2a8b16f4c1c01d3b3a26"
server_key_version: "encryptionkey@0"
I20260806 15:07:26.950929 2095 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:07:26.951592 2123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:26.951749 2095 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:07:26.951838 2095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-0/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-0/wal
uuid: "a735dfc7faf44634a35995bea09a8201"
format_stamp: "Formatted at 2026-08-06 15:07:26 on dist-test-slave-tc2s"
server_key: "bbf26989465974ae9fee84d13b231f8c"
server_key_iv: "20669d509fbe2a8b16f4c1c01d3b3a26"
server_key_version: "encryptionkey@0"
I20260806 15:07:26.951898 2095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:07:26.965852 2095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:07:26.967653 2095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:07:26.967793 2095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:07:26.971602 2095 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.126:46183
I20260806 15:07:26.971638 2202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.126:46183 every 8 connection(s)
I20260806 15:07:26.971946 2095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-0/data/info.pb
I20260806 15:07:26.972884 2204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:07:26.974483 2204 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46183 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:07:26.974992 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 2095
I20260806 15:07:26.975080 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-0/wal/instance
I20260806 15:07:26.975265 945 external_mini_cluster.cc:1428] Setting key 91d843a36c735e84b5c4aefb110935a6
Aug 06 15:07:26 dist-test-slave-tc2s krb5kdc[2071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028846, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.236.125@KRBTEST.COM; defaulting to no policy
I20260806 15:07:26.985082 2206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:26.975167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:46183 (local address 127.0.0.1:48214)
0806 15:07:26.975338 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:26.975388 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 15:07:26.975970 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:07:26.976234 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:07:26.976237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:07:26.976581 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:07:26.976804 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:26.976810 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:26.978251 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:26.978252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:26.979224 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:26.979226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:26.979727 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:26.979729 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:26.979817 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:26.980334 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:07:26.980344 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:07:26.981969 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:07:26.983614 (+ 1645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:26.983622 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:26.983624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:26.983878 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:26.984094 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:26.984095 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:26.984095 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:26.984205 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:26.984596 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:07:26.984603 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:07:26.984665 (+ 62us) client_negotiation.cc:769] Sending connection context
0806 15:07:26.984743 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 15:07:26.984947 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":77,"thread_start_us":83,"threads_started":1}
I20260806 15:07:26.985107 2207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:26.975230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48214 (local address 127.0.236.126:46183)
0806 15:07:26.975400 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:26.975405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:26.975494 (+ 89us) server_negotiation.cc:407] Connection header received
0806 15:07:26.976108 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:26.976109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:26.976135 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:26.976209 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:26.976887 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:26.978158 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:26.979303 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:26.979615 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:26.979693 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:26.982052 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:26.982060 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:26.982060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:26.982067 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:26.983527 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:26.983950 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:26.983952 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:26.983952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:26.984038 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:26.984291 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:26.984292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:26.984292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:26.984534 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:26.984598 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:26.984780 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 15:07:26.984875 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":57,"threads_started":1}
I20260806 15:07:26.985847 2204 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.125" port: 40801 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:07:26.986209 2206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:26.986106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:40801 (local address 127.0.0.1:40242)
0806 15:07:26.986146 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:26.986191 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.125:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 15:07:26.986325 2124 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.236.125:40801: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.125@KRBTEST.COM" created.
I20260806 15:07:26.986696 2206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:26.986630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:40801 (local address 127.0.0.1:40256)
0806 15:07:26.986658 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:26.986680 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.125:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260806 15:07:26.986788 2204 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.236.125:40801: Network error: Client connection negotiation failed: client connection to 127.0.236.125:40801: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/krb5kdc/kudu_127.0.236.125.keytab.
Entry for principal kudu/127.0.236.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/krb5kdc/kudu_127.0.236.125.keytab.
I20260806 15:07:27.000736 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.125:40801
--webserver_interface=127.0.236.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/krb5kdc/kudu_127.0.236.125.keytab
--principal=kudu/127.0.236.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:43053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:46183,127.0.236.125:40801,127.0.236.124:33191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:07:27.036038 2204 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.236.125" port: 40801 } attempt: 1
I20260806 15:07:27.036567 2206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.036441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:40801 (local address 127.0.0.1:40258)
0806 15:07:27.036498 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:27.036531 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.125:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 15:07:27.036949 2206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.036808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:40801 (local address 127.0.0.1:40266)
0806 15:07:27.036868 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:27.036914 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.125:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 15:07:27.037066 2204 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.236.125:40801: Network error: Client connection negotiation failed: client connection to 127.0.236.125:40801: connect: Connection refused (error 111)
W20260806 15:07:27.081573 2213 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 15:07:27.081787 2213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:07:27.081804 2213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:07:27.081822 2213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:07:27.082960 2213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:07:27.083004 2213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:07:27.083024 2213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:07:27.083035 2213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:07:27.083050 2213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:07:27.084509 2213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:43053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:46183,127.0.236.125:40801,127.0.236.124:33191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/krb5kdc/kudu_127.0.236.125.keytab
--principal=kudu/127.0.236.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.125:40801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.236.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:07:27.084708 2213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:07:27.084880 2213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:07:27.089514 2222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:27.089521 2220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:27.089521 2225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:27.089859 2213 server_base.cc:1034] running on GCE node
I20260806 15:07:27.090164 2213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:07:27.090418 2213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:07:27.091566 2213 hybrid_clock.cc:648] HybridClock initialized: now 1786028847091552 us; error 27 us; skew 500 ppm
Aug 06 15:07:27 dist-test-slave-tc2s krb5kdc[2071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028847, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:07:27.093977 2213 init.cc:377] Logged in from keytab as kudu/127.0.236.125@KRBTEST.COM (short username kudu)
I20260806 15:07:27.094400 2213 webserver.cc:460] Webserver started at http://127.0.236.125:33545/ using document root <none> and password file <none>
I20260806 15:07:27.094604 2213 fs_manager.cc:359] Metadata directory not provided
I20260806 15:07:27.094650 2213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:07:27.094771 2213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:07:27.095829 2213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-1/data/instance:
uuid: "39171c3b03544bf5b291757bc48b7021"
format_stamp: "Formatted at 2026-08-06 15:07:27 on dist-test-slave-tc2s"
server_key: "cf7d9c33ffb356b9bf29d85ef45290d7"
server_key_iv: "ee556b313235e060c776fbd6be3f1bdc"
server_key_version: "encryptionkey@0"
I20260806 15:07:27.096165 2213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-1/wal/instance:
uuid: "39171c3b03544bf5b291757bc48b7021"
format_stamp: "Formatted at 2026-08-06 15:07:27 on dist-test-slave-tc2s"
server_key: "cf7d9c33ffb356b9bf29d85ef45290d7"
server_key_iv: "ee556b313235e060c776fbd6be3f1bdc"
server_key_version: "encryptionkey@0"
I20260806 15:07:27.097354 2213 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:07:27.097999 2232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:27.098132 2213 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:27.098217 2213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-1/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-1/wal
uuid: "39171c3b03544bf5b291757bc48b7021"
format_stamp: "Formatted at 2026-08-06 15:07:27 on dist-test-slave-tc2s"
server_key: "cf7d9c33ffb356b9bf29d85ef45290d7"
server_key_iv: "ee556b313235e060c776fbd6be3f1bdc"
server_key_version: "encryptionkey@0"
I20260806 15:07:27.098281 2213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:07:27.105202 2204 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.236.125" port: 40801 } attempt: 2
I20260806 15:07:27.105567 2206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.105454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:40801 (local address 127.0.0.1:40270)
0806 15:07:27.105505 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:27.105540 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.125:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 15:07:27.105849 2206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.105786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:40801 (local address 127.0.0.1:40282)
0806 15:07:27.105813 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:27.105834 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.125:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260806 15:07:27.105968 2204 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.236.125:40801: Network error: Client connection negotiation failed: client connection to 127.0.236.125:40801: connect: Connection refused (error 111)
I20260806 15:07:27.109609 2213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:07:27.111611 2213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:07:27.111735 2213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:07:27.115561 2213 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.125:40801
I20260806 15:07:27.115600 2307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.125:40801 every 8 connection(s)
I20260806 15:07:27.115904 2213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-1/data/info.pb
I20260806 15:07:27.116832 2308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:07:27.118468 2308 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46183 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 15:07:27 dist-test-slave-tc2s krb5kdc[2071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028847, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.125@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:07:27.125630 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 2213
I20260806 15:07:27.125743 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-1/wal/instance
I20260806 15:07:27.125963 945 external_mini_cluster.cc:1428] Setting key e557b619d5997c939503f274de78bafd
I20260806 15:07:27.126807 2207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.119155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48216 (local address 127.0.236.126:46183)
0806 15:07:27.119227 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.119230 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.119385 (+ 155us) server_negotiation.cc:407] Connection header received
0806 15:07:27.119840 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.119841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.119855 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.119895 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.120510 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.120986 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.121877 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.122111 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.122162 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:27.124204 (+ 2042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:27.124205 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:27.124206 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:27.124213 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:27.125348 (+ 1135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.125819 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.125821 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.125821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.125885 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.126203 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.126204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.126205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.126322 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:27.126370 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:27.126672 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 15:07:27.126742 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 15:07:27.127136 2310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.119122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:46183 (local address 127.0.0.1:48216)
0806 15:07:27.119276 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:27.119312 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 15:07:27.119752 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:07:27.119974 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:07:27.119977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:07:27.120223 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:07:27.120440 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:27.120444 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.121041 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.121042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:27.121818 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:27.121820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.122187 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.122187 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:27.122290 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:27.122725 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:07:27.122732 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:07:27.124114 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:07:27.125417 (+ 1303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:27.125427 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:27.125429 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:27.125728 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:27.125983 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:27.125984 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:27.125985 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:27.126132 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:27.126377 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:07:27.126383 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:07:27.126543 (+ 160us) client_negotiation.cc:769] Sending connection context
0806 15:07:27.126650 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 15:07:27.126815 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":78,"threads_started":1}
I20260806 15:07:27.127844 2308 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.125" port: 40801 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 15:07:27 dist-test-slave-tc2s krb5kdc[2071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028847, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.125@KRBTEST.COM for kudu/127.0.236.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.236.124@KRBTEST.COM; defaulting to no policy
I20260806 15:07:27.134933 2310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.128091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:40801 (local address 127.0.0.1:40294)
0806 15:07:27.128268 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:27.128279 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:07:27.128335 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:07:27.128463 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:07:27.128464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:07:27.128586 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:07:27.128631 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:27.128633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.129463 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.129464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:27.130010 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:27.130012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.130660 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.130661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:27.130717 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:27.131083 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:07:27.131088 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:07:27.132170 (+ 1082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:07:27.134000 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:27.134001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:27.134001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:27.134150 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:27.134370 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:27.134371 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:27.134371 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:27.134424 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:27.134688 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:07:27.134689 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:07:27.134730 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 15:07:27.134783 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 15:07:27.134867 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 15:07:27.135041 2316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.128154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40294 (local address 127.0.236.125:40801)
0806 15:07:27.128289 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.128293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.128348 (+ 55us) server_negotiation.cc:407] Connection header received
0806 15:07:27.128381 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.128384 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.128396 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.128456 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.128695 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.129387 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.130097 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.130588 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.130663 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:27.132249 (+ 1586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:27.132258 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:27.132258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:27.132265 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:27.133922 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.134221 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.134225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.134225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.134294 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.134486 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.134487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.134487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.134630 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:27.134690 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:27.134807 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 15:07:27.134932 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
I20260806 15:07:27.135553 2308 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:07:27.135862 2310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.135776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:33191 (local address 127.0.0.1:59096)
0806 15:07:27.135806 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:27.135847 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:33191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260806 15:07:27.135982 2235 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.236.124:33191: connect: Connection refused (error 111)
I20260806 15:07:27.136402 2310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.136308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:33191 (local address 127.0.0.1:59100)
0806 15:07:27.136351 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:27.136381 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:33191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 15:07:27.136497 2308 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.236.124:33191: Network error: Client connection negotiation failed: client connection to 127.0.236.124:33191: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.236.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.236.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/krb5kdc/kudu_127.0.236.124.keytab.
Entry for principal kudu/127.0.236.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/krb5kdc/kudu_127.0.236.124.keytab.
I20260806 15:07:27.151084 945 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFIWuwX/build/release/bin/kudu
/tmp/dist-test-taskFIWuwX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.124:33191
--webserver_interface=127.0.236.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/krb5kdc/kudu_127.0.236.124.keytab
--principal=kudu/127.0.236.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.236.84:43053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.236.126:46183,127.0.236.125:40801,127.0.236.124:33191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:07:27.185716 2308 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 } attempt: 1
I20260806 15:07:27.186257 2310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.186086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:33191 (local address 127.0.0.1:59114)
0806 15:07:27.186158 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:27.186213 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:33191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 15:07:27.186617 2310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.186513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:33191 (local address 127.0.0.1:59116)
0806 15:07:27.186560 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:27.186593 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:33191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 15:07:27.186718 2308 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.236.124:33191: Network error: Client connection negotiation failed: client connection to 127.0.236.124:33191: connect: Connection refused (error 111)
I20260806 15:07:27.197113 2204 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.236.125" port: 40801 } attempt: 3
Aug 06 15:07:27 dist-test-slave-tc2s krb5kdc[2071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028846, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for kudu/127.0.236.125@KRBTEST.COM
I20260806 15:07:27.204005 2316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.197522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40302 (local address 127.0.236.125:40801)
0806 15:07:27.197584 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.197586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.197595 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:07:27.197620 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.197622 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.197633 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.197677 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.198029 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.198540 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.199233 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.199449 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.199506 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:27.201618 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:27.201619 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:27.201619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:27.201626 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:27.202988 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.203301 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.203302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.203303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.203357 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.203522 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.203523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.203524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.203633 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:27.203690 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:27.203832 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 15:07:27.203906 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 15:07:27.204010 2206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.197419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:40801 (local address 127.0.0.1:40302)
0806 15:07:27.197487 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:27.197498 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:07:27.197542 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:07:27.197691 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:07:27.197692 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:07:27.197855 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:07:27.197953 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:27.197956 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.198622 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.198623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:27.199157 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:27.199159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.199512 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.199513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:27.199592 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:27.200104 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:07:27.200109 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:07:27.201526 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:07:27.203070 (+ 1544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:27.203071 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:27.203071 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:27.203240 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:27.203410 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:27.203411 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:27.203411 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:27.203472 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:27.203688 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:07:27.203689 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:07:27.203742 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 15:07:27.203800 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 15:07:27.203906 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 15:07:27.204460 2204 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:07:27.204811 2206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.204714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:33191 (local address 127.0.0.1:59128)
0806 15:07:27.204760 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:27.204790 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:33191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 15:07:27.205453 2206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.205369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:33191 (local address 127.0.0.1:59144)
0806 15:07:27.205400 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:27.205425 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:33191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260806 15:07:27.205538 2204 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.236.124:33191: Network error: Client connection negotiation failed: client connection to 127.0.236.124:33191: connect: Connection refused (error 111)
W20260806 15:07:27.234099 2320 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 15:07:27.234304 2320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:07:27.234328 2320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:07:27.234352 2320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:07:27.235456 2320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:07:27.235496 2320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:07:27.235514 2320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:07:27.235528 2320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:07:27.235543 2320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:07:27.236675 2204 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 } attempt: 1
I20260806 15:07:27.237033 2320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.236.84:43053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.236.126:46183,127.0.236.125:40801,127.0.236.124:33191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/krb5kdc/kudu_127.0.236.124.keytab
--principal=kudu/127.0.236.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.236.124:33191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.236.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:07:27.237058 2206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.236946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:33191 (local address 127.0.0.1:59160)
0806 15:07:27.236998 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:27.237030 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:33191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 15:07:27.237272 2320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:07:27.237383 2206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.237294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:33191 (local address 127.0.0.1:59172)
0806 15:07:27.237334 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:27.237360 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:33191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 15:07:27.237471 2204 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.236.124:33191: Network error: Client connection negotiation failed: client connection to 127.0.236.124:33191: connect: Connection refused (error 111)
I20260806 15:07:27.237486 2320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 15:07:27.242005 2320 server_base.cc:1034] running on GCE node
W20260806 15:07:27.241979 2329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:27.241982 2328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:07:27.241976 2331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:07:27.242359 2320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:07:27.242643 2320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:07:27.243775 2320 hybrid_clock.cc:648] HybridClock initialized: now 1786028847243766 us; error 28 us; skew 500 ppm
Aug 06 15:07:27 dist-test-slave-tc2s krb5kdc[2071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028847, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:07:27.246172 2320 init.cc:377] Logged in from keytab as kudu/127.0.236.124@KRBTEST.COM (short username kudu)
I20260806 15:07:27.246587 2320 webserver.cc:460] Webserver started at http://127.0.236.124:46029/ using document root <none> and password file <none>
I20260806 15:07:27.246778 2320 fs_manager.cc:359] Metadata directory not provided
I20260806 15:07:27.246819 2320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:07:27.246937 2320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:07:27.248167 2320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-2/data/instance:
uuid: "96874dfb5bd4435bad58750969ef9c8e"
format_stamp: "Formatted at 2026-08-06 15:07:27 on dist-test-slave-tc2s"
server_key: "ee8985ac29d6cfc89e3f95da32d5b3f0"
server_key_iv: "5a23be5feb9c26d664a6fd0554ded3a3"
server_key_version: "encryptionkey@0"
I20260806 15:07:27.248490 2320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-2/wal/instance:
uuid: "96874dfb5bd4435bad58750969ef9c8e"
format_stamp: "Formatted at 2026-08-06 15:07:27 on dist-test-slave-tc2s"
server_key: "ee8985ac29d6cfc89e3f95da32d5b3f0"
server_key_iv: "5a23be5feb9c26d664a6fd0554ded3a3"
server_key_version: "encryptionkey@0"
I20260806 15:07:27.249784 2320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:07:27.250517 2341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:27.250648 2320 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:07:27.250725 2320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-2/data,/tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-2/wal
uuid: "96874dfb5bd4435bad58750969ef9c8e"
format_stamp: "Formatted at 2026-08-06 15:07:27 on dist-test-slave-tc2s"
server_key: "ee8985ac29d6cfc89e3f95da32d5b3f0"
server_key_iv: "5a23be5feb9c26d664a6fd0554ded3a3"
server_key_version: "encryptionkey@0"
I20260806 15:07:27.250790 2320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:07:27.254845 2308 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 } attempt: 2
I20260806 15:07:27.255189 2310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.255079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:33191 (local address 127.0.0.1:59180)
0806 15:07:27.255128 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:27.255159 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:33191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 15:07:27.255510 2310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.255431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:33191 (local address 127.0.0.1:59194)
0806 15:07:27.255465 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:27.255489 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.236.124:33191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 15:07:27.255591 2308 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.236.124:33191: Network error: Client connection negotiation failed: client connection to 127.0.236.124:33191: connect: Connection refused (error 111)
I20260806 15:07:27.258198 2320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:07:27.260111 2320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:07:27.260236 2320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:07:27.265635 2320 rpc_server.cc:307] RPC server started. Bound to: 127.0.236.124:33191
I20260806 15:07:27.265666 2413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.236.124:33191 every 8 connection(s)
I20260806 15:07:27.265995 2320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-2/data/info.pb
I20260806 15:07:27.266137 945 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFIWuwX/build/release/bin/kudu as pid 2320
I20260806 15:07:27.266252 945 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFIWuwX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028580791223-945-0/minicluster-data/master-2/wal/instance
I20260806 15:07:27.266453 945 external_mini_cluster.cc:1428] Setting key c4a3af8603fce5e2b415bff018ff99da
I20260806 15:07:27.267127 2414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:07:27.268914 2414 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46183 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 15:07:27 dist-test-slave-tc2s krb5kdc[2071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.124@KRBTEST.COM
Aug 06 15:07:27 dist-test-slave-tc2s krb5kdc[2071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028847, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.124@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:07:27.276000 2417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.266892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59206 (local address 127.0.236.124:33191)
0806 15:07:27.267058 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.267067 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.267096 (+ 29us) server_negotiation.cc:407] Connection header received
0806 15:07:27.267524 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.267531 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.267584 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.267693 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.268486 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.269387 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.270242 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.270647 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.270720 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:27.273062 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:27.273070 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:27.273070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:27.273076 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:27.274548 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.274865 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.274868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.274868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.274946 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.275116 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.275117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.275117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.275402 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:27.275534 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:27.275659 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 15:07:27.275780 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
I20260806 15:07:27.276192 2418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.269453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.126:46183 (local address 127.0.0.1:48232)
0806 15:07:27.269614 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:27.269648 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 15:07:27.269703 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:07:27.269844 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:07:27.269846 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:07:27.270123 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:07:27.270185 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:27.270187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.270822 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.270823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:27.271491 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:27.271494 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.271968 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.271969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:27.272015 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:27.272390 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:07:27.272400 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:07:27.273774 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:07:27.275158 (+ 1384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:27.275163 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:27.275165 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:27.275390 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:27.275579 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:27.275580 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:27.275580 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:27.275649 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:27.275878 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:07:27.275883 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:07:27.275932 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 15:07:27.276017 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 15:07:27.276116 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260806 15:07:27.276196 2207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.269513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48232 (local address 127.0.236.126:46183)
0806 15:07:27.269556 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.269557 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.269702 (+ 145us) server_negotiation.cc:407] Connection header received
0806 15:07:27.269762 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.269762 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.269773 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.269820 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.270268 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.270736 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.271574 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.271874 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.271948 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:27.273860 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:27.273861 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:27.273861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:27.273868 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:27.275105 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.275460 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.275461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.275461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.275516 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.275713 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.275714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.275714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.275830 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:27.275880 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:27.276035 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 15:07:27.276111 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 15:07:27.276707 2414 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.125" port: 40801 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 15:07:27 dist-test-slave-tc2s krb5kdc[2071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028847, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.124@KRBTEST.COM for kudu/127.0.236.125@KRBTEST.COM
Aug 06 15:07:27 dist-test-slave-tc2s krb5kdc[2071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.126@KRBTEST.COM
I20260806 15:07:27.282929 2418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.276897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.125:40801 (local address 127.0.0.1:40304)
0806 15:07:27.276984 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:27.276994 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:07:27.277029 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:07:27.277147 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:07:27.277148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:07:27.277247 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:07:27.277295 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:27.277296 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.278086 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.278087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:27.278661 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:27.278663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.279077 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.279078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:27.279142 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:27.279515 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:07:27.279519 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:07:27.280563 (+ 1044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:07:27.282068 (+ 1505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:27.282069 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:27.282069 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:27.282226 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:27.282390 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:27.282390 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:27.282390 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:27.282446 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:27.282652 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:07:27.282652 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:07:27.282692 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 15:07:27.282751 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 15:07:27.282833 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 15:07:27.282927 2316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.276988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40304 (local address 127.0.236.125:40801)
0806 15:07:27.277048 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.277050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.277059 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:07:27.277083 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.277084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.277093 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.277143 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.277361 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.278027 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.278727 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.279018 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.279074 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:27.280640 (+ 1566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:27.280641 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:27.280641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:27.280647 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:27.282004 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.282283 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.282284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.282285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.282338 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.282494 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.282495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.282495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.282599 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:27.282652 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:27.282779 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 15:07:27.282840 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 15:07:27.283208 2414 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:07:27.283568 2207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.276916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48242 (local address 127.0.236.126:46183)
0806 15:07:27.276950 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.276951 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.277170 (+ 219us) server_negotiation.cc:407] Connection header received
0806 15:07:27.277184 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.277185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.277191 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.277229 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.277737 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.278291 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.278995 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.279220 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.279282 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:27.281165 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:27.281166 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:27.281166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:27.281173 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:27.282424 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.282707 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.282708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.282709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.282755 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.282921 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.282922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.282922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.283034 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:27.283208 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:27.283404 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 15:07:27.283484 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Aug 06 15:07:27 dist-test-slave-tc2s krb5kdc[2071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028847, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.124@KRBTEST.COM for kudu/127.0.236.124@KRBTEST.COM
Aug 06 15:07:27 dist-test-slave-tc2s krb5kdc[2071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.236.125@KRBTEST.COM
I20260806 15:07:27.289419 2418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.283403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:33191 (local address 127.0.0.1:59210)
0806 15:07:27.283532 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:27.283540 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 15:07:27.283581 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:07:27.283785 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:07:27.283786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:07:27.283918 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:07:27.283949 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:27.283950 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.284611 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.284613 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:27.285085 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:27.285087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.285394 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.285395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:27.285446 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:27.285736 (+ 290us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:07:27.285740 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:07:27.287027 (+ 1287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:07:27.288460 (+ 1433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:27.288461 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:27.288461 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:27.288654 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:27.288840 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:27.288841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:27.288841 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:27.288906 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:27.289123 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:07:27.289123 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:07:27.289166 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 15:07:27.289232 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 15:07:27.289318 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 15:07:27.289466 2417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.283499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59210 (local address 127.0.236.124:33191)
0806 15:07:27.283532 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.283534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.283569 (+ 35us) server_negotiation.cc:407] Connection header received
0806 15:07:27.283712 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.283713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.283722 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.283775 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.284017 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.284542 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.285157 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.285338 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.285381 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:27.287102 (+ 1721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:27.287103 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:27.287103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:27.287109 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:27.288395 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.288718 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.288719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.288720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.288780 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.288960 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.288960 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.288961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.289065 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:27.289126 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:27.289262 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 15:07:27.289380 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":18}
I20260806 15:07:27.290738 2316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.284129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40314 (local address 127.0.236.125:40801)
0806 15:07:27.284166 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.284168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.284175 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:07:27.284192 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.284193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.284200 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.284257 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.284627 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.285040 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.285781 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.286009 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.286077 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:27.288390 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:27.288391 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:27.288391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:27.288397 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:27.289787 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.290127 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.290128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.290128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.290181 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.290354 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.290355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.290355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.290444 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:27.290544 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:27.290624 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 15:07:27.290680 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 15:07:27.291270 2414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96874dfb5bd4435bad58750969ef9c8e: Bootstrap starting.
I20260806 15:07:27.291988 945 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:07:27.292200 2414 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96874dfb5bd4435bad58750969ef9c8e: Neither blocks nor log segments found. Creating new log.
I20260806 15:07:27.292455 2414 log.cc:824] T 00000000000000000000000000000000 P 96874dfb5bd4435bad58750969ef9c8e: Log is configured to *not* fsync() on all Append() calls
I20260806 15:07:27.293223 2414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96874dfb5bd4435bad58750969ef9c8e: No bootstrap required, opened a new log
I20260806 15:07:27.292816 2160 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a735dfc7faf44634a35995bea09a8201"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48242
I20260806 15:07:27.294675 2414 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96874dfb5bd4435bad58750969ef9c8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a735dfc7faf44634a35995bea09a8201" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46183 } } peers { permanent_uuid: "39171c3b03544bf5b291757bc48b7021" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 40801 } } peers { permanent_uuid: "96874dfb5bd4435bad58750969ef9c8e" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 } }
I20260806 15:07:27.294791 2414 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96874dfb5bd4435bad58750969ef9c8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96874dfb5bd4435bad58750969ef9c8e, State: Initialized, Role: FOLLOWER
I20260806 15:07:27.294873 2414 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96874dfb5bd4435bad58750969ef9c8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a735dfc7faf44634a35995bea09a8201" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46183 } } peers { permanent_uuid: "39171c3b03544bf5b291757bc48b7021" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 40801 } } peers { permanent_uuid: "96874dfb5bd4435bad58750969ef9c8e" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 } }
I20260806 15:07:27.295127 2421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96874dfb5bd4435bad58750969ef9c8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a735dfc7faf44634a35995bea09a8201" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46183 } } peers { permanent_uuid: "39171c3b03544bf5b291757bc48b7021" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 40801 } } peers { permanent_uuid: "96874dfb5bd4435bad58750969ef9c8e" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 } } }
I20260806 15:07:27.295174 2414 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96874dfb5bd4435bad58750969ef9c8e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:07:27.295220 2421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96874dfb5bd4435bad58750969ef9c8e [sys.catalog]: This master's current role is: FOLLOWER
W20260806 15:07:27.296891 2434 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 96874dfb5bd4435bad58750969ef9c8e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:07:27.296945 2434 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:07:27.297158 2320 master_runner.cc:186] Error getting master registration for 127.0.236.126:46183: OK, instance_id { permanent_uuid: "a735dfc7faf44634a35995bea09a8201" instance_seqno: 1786028846970590 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 15:07:27.299849 2320 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 15:07:27.312608 2204 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 } attempt: 2
Aug 06 15:07:27 dist-test-slave-tc2s krb5kdc[2071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028846, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.126@KRBTEST.COM for kudu/127.0.236.124@KRBTEST.COM
I20260806 15:07:27.319051 2206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.312879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:33191 (local address 127.0.0.1:59220)
0806 15:07:27.312933 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:27.312960 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 15:07:27.313023 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:07:27.313136 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:07:27.313137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:07:27.313291 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:07:27.313332 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:27.313334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.313935 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.313936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:27.314435 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:27.314436 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.314743 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.314743 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:27.314796 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:27.315243 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:07:27.315247 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:07:27.316501 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:07:27.318115 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:27.318116 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:27.318116 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:27.318276 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:27.318448 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:27.318448 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:27.318449 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:27.318506 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:27.318742 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:07:27.318742 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:07:27.318789 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 15:07:27.318856 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 15:07:27.318952 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 15:07:27.319051 2417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.312934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59220 (local address 127.0.236.124:33191)
0806 15:07:27.312971 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.312973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.313007 (+ 34us) server_negotiation.cc:407] Connection header received
0806 15:07:27.313072 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.313072 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.313081 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.313127 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.313403 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.313846 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.314508 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.314686 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.314739 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:27.316576 (+ 1837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:27.316578 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:27.316578 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:27.316585 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:27.318041 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.318333 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.318335 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.318335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.318397 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.318554 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.318555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.318555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.318685 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:27.318743 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:27.318877 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 15:07:27.318952 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:07:27.320626 2204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201: Bootstrap starting.
I20260806 15:07:27.321208 2204 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201: Neither blocks nor log segments found. Creating new log.
I20260806 15:07:27.321444 2204 log.cc:824] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201: Log is configured to *not* fsync() on all Append() calls
I20260806 15:07:27.322177 2204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201: No bootstrap required, opened a new log
I20260806 15:07:27.323148 2204 raft_consensus.cc:348] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a735dfc7faf44634a35995bea09a8201" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46183 } } peers { permanent_uuid: "39171c3b03544bf5b291757bc48b7021" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 40801 } } peers { permanent_uuid: "96874dfb5bd4435bad58750969ef9c8e" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 } }
I20260806 15:07:27.323256 2204 raft_consensus.cc:729] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a735dfc7faf44634a35995bea09a8201, State: Initialized, Role: FOLLOWER
I20260806 15:07:27.323318 2204 consensus_queue.cc:260] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a735dfc7faf44634a35995bea09a8201" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46183 } } peers { permanent_uuid: "39171c3b03544bf5b291757bc48b7021" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 40801 } } peers { permanent_uuid: "96874dfb5bd4435bad58750969ef9c8e" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 } }
I20260806 15:07:27.323599 2204 sys_catalog.cc:564] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:07:27.323560 2438 sys_catalog.cc:455] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a735dfc7faf44634a35995bea09a8201" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46183 } } peers { permanent_uuid: "39171c3b03544bf5b291757bc48b7021" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 40801 } } peers { permanent_uuid: "96874dfb5bd4435bad58750969ef9c8e" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 } } }
I20260806 15:07:27.323657 2438 sys_catalog.cc:458] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:07:27.323683 2160 raft_consensus.cc:482] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 15:07:27.323823 2160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:07:27.324656 2160 raft_consensus.cc:504] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a735dfc7faf44634a35995bea09a8201" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46183 } } peers { permanent_uuid: "39171c3b03544bf5b291757bc48b7021" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 40801 } } peers { permanent_uuid: "96874dfb5bd4435bad58750969ef9c8e" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 } }
I20260806 15:07:27.325434 2377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a735dfc7faf44634a35995bea09a8201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "96874dfb5bd4435bad58750969ef9c8e"
I20260806 15:07:27.325551 2377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96874dfb5bd4435bad58750969ef9c8e [term 0 FOLLOWER]: Advancing to term 1
W20260806 15:07:27.325773 2453 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:07:27.325811 2453 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:07:27.326376 2377 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 96874dfb5bd4435bad58750969ef9c8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a735dfc7faf44634a35995bea09a8201 in term 1.
I20260806 15:07:27.327011 2127 leader_election.cc:302] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96874dfb5bd4435bad58750969ef9c8e, a735dfc7faf44634a35995bea09a8201; no voters:
I20260806 15:07:27.327122 2438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:07:27.327250 2438 raft_consensus.cc:686] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201 [term 1 LEADER]: Becoming Leader. State: Replica: a735dfc7faf44634a35995bea09a8201, State: Running, Role: LEADER
I20260806 15:07:27.327324 2438 consensus_queue.cc:237] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a735dfc7faf44634a35995bea09a8201" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46183 } } peers { permanent_uuid: "39171c3b03544bf5b291757bc48b7021" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 40801 } } peers { permanent_uuid: "96874dfb5bd4435bad58750969ef9c8e" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 } }
I20260806 15:07:27.327699 2438 sys_catalog.cc:455] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a735dfc7faf44634a35995bea09a8201. Latest consensus state: current_term: 1 leader_uuid: "a735dfc7faf44634a35995bea09a8201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a735dfc7faf44634a35995bea09a8201" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46183 } } peers { permanent_uuid: "39171c3b03544bf5b291757bc48b7021" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 40801 } } peers { permanent_uuid: "96874dfb5bd4435bad58750969ef9c8e" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 } } }
I20260806 15:07:27.327762 2438 sys_catalog.cc:458] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201 [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:27.328094 2160 leader_election.cc:288] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201 [CANDIDATE]: Term 1 election: Requested vote from peers 39171c3b03544bf5b291757bc48b7021 (127.0.236.125:40801), 96874dfb5bd4435bad58750969ef9c8e (127.0.236.124:33191)
I20260806 15:07:27.329205 2457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:07:27.327368 2095 master_runner.cc:186] Error getting master registration for 127.0.236.125:40801: OK, instance_id { permanent_uuid: "39171c3b03544bf5b291757bc48b7021" instance_seqno: 1786028847114479 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 15:07:27.329607 2095 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 15:07:27.329685 2457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:07:27.331700 2377 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 96874dfb5bd4435bad58750969ef9c8e [term 1 FOLLOWER]: Refusing update from remote peer a735dfc7faf44634a35995bea09a8201: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:07:27.332005 2438 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96874dfb5bd4435bad58750969ef9c8e" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:07:27.333201 2421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96874dfb5bd4435bad58750969ef9c8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a735dfc7faf44634a35995bea09a8201. Latest consensus state: current_term: 1 leader_uuid: "a735dfc7faf44634a35995bea09a8201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a735dfc7faf44634a35995bea09a8201" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46183 } } peers { permanent_uuid: "39171c3b03544bf5b291757bc48b7021" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 40801 } } peers { permanent_uuid: "96874dfb5bd4435bad58750969ef9c8e" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 } } }
I20260806 15:07:27.333292 2421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96874dfb5bd4435bad58750969ef9c8e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:07:27.334246 2438 sys_catalog.cc:455] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a735dfc7faf44634a35995bea09a8201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a735dfc7faf44634a35995bea09a8201" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46183 } } peers { permanent_uuid: "39171c3b03544bf5b291757bc48b7021" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 40801 } } peers { permanent_uuid: "96874dfb5bd4435bad58750969ef9c8e" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 } } }
I20260806 15:07:27.334321 2438 sys_catalog.cc:458] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201 [sys.catalog]: This master's current role is: LEADER
I20260806 15:07:27.334438 2462 mvcc.cc:205] Tried to move back new op lower bound from 7315574158667931648 to 7315574158653341696. Current Snapshot: MvccSnapshot[applied={T|T < 7315574158667931648}]
I20260806 15:07:27.334444 2421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96874dfb5bd4435bad58750969ef9c8e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a735dfc7faf44634a35995bea09a8201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a735dfc7faf44634a35995bea09a8201" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46183 } } peers { permanent_uuid: "39171c3b03544bf5b291757bc48b7021" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 40801 } } peers { permanent_uuid: "96874dfb5bd4435bad58750969ef9c8e" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 } } }
I20260806 15:07:27.334519 2421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96874dfb5bd4435bad58750969ef9c8e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:07:27.334735 2465 mvcc.cc:205] Tried to move back new op lower bound from 7315574158667931648 to 7315574158653341696. Current Snapshot: MvccSnapshot[applied={T|T < 7315574158667931648}]
I20260806 15:07:27.338315 2457 catalog_manager.cc:1347] Generated new cluster ID: 257147ba73bc41dfbd6c100709cfc3ae
I20260806 15:07:27.338362 2457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:07:27.346714 2308 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 } attempt: 3
I20260806 15:07:27.349237 2457 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:07:27.350862 2457 catalog_manager.cc:1504] Loading token signing keys...
Aug 06 15:07:27 dist-test-slave-tc2s krb5kdc[2071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028847, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.236.125@KRBTEST.COM for kudu/127.0.236.124@KRBTEST.COM
I20260806 15:07:27.353502 2417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.331144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59230 (local address 127.0.236.124:33191)
0806 15:07:27.331188 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.331190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.342066 (+ 10876us) server_negotiation.cc:407] Connection header received
0806 15:07:27.342110 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.342111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.342121 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.342149 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.342919 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.343626 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.344679 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.344940 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.345002 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:27.348583 (+ 3581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:27.348584 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:27.348585 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:27.348592 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:27.350102 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.350474 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.350476 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.350476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.350534 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.352006 (+ 1472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.352007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.352008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.352140 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:27.352209 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:27.353316 (+ 1107us) server_negotiation.cc:299] Negotiation successful
0806 15:07:27.353390 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 15:07:27.358309 2207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.332136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48252 (local address 127.0.236.126:46183)
0806 15:07:27.332183 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.332185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.343003 (+ 10818us) server_negotiation.cc:407] Connection header received
0806 15:07:27.343087 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.343088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.343098 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.343124 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.343777 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.344472 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.345775 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.346414 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.346476 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:27.353943 (+ 7467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:27.353945 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:27.353945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:27.353952 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:27.355568 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.357192 (+ 1624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.357193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.357193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.357250 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.357441 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.357442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.357442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.357564 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:27.357630 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:27.358125 (+ 495us) server_negotiation.cc:299] Negotiation successful
0806 15:07:27.358219 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:07:27.360114 2457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201: Generated new TSK 0
I20260806 15:07:27.360239 2316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.332592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40326 (local address 127.0.236.125:40801)
0806 15:07:27.332632 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.332634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.344748 (+ 12114us) server_negotiation.cc:407] Connection header received
0806 15:07:27.344833 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.344834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.344844 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.344871 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.346673 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.347368 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.350740 (+ 3372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.351027 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.351070 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:27.354232 (+ 3162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:27.354234 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:27.354234 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:27.354241 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:27.356733 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.359174 (+ 2441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.359175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.359176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.359237 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.359442 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.359443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.359443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.359583 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:27.359686 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:27.360067 (+ 381us) server_negotiation.cc:299] Negotiation successful
0806 15:07:27.360145 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 15:07:27.360684 2457 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:07:27.362941 2310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.346966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.236.124:33191 (local address 127.0.0.1:59234)
0806 15:07:27.347006 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 15:07:27.347016 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:07:27.347065 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:07:27.348295 (+ 1230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:07:27.348296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:07:27.348462 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:07:27.348505 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:27.348507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.349511 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.349513 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:27.350325 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:07:27.350328 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.350807 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.350808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:07:27.350874 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:27.351247 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:07:27.351252 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:07:27.352705 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:07:27.361671 (+ 8966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:27.361672 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:27.361673 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:27.361910 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:27.362144 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:07:27.362145 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:07:27.362145 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:07:27.362219 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:07:27.362651 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:07:27.362652 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:07:27.362709 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 15:07:27.362755 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 15:07:27.362845 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 15:07:27.363623 2473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:27.347063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59234 (local address 127.0.236.124:33191)
0806 15:07:27.348137 (+ 1074us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.348139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.348147 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:07:27.348164 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.348165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.348174 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.348588 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.348632 (+ 44us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.349406 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.350421 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:27.350730 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:27.351017 (+ 287us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:27.352889 (+ 1872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:07:27.352890 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:07:27.352891 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:07:27.352897 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:07:27.361579 (+ 8682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.361997 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.361998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.361998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.362060 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:07:27.362303 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:07:27.362304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:07:27.362305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:07:27.362551 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:07:27.363221 (+ 670us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:27.363305 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 15:07:27.363443 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1053,"thread_start_us":87,"threads_started":1}
I20260806 15:07:27.365298 2308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39171c3b03544bf5b291757bc48b7021: Bootstrap starting.
I20260806 15:07:27.366066 2308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39171c3b03544bf5b291757bc48b7021: Neither blocks nor log segments found. Creating new log.
I20260806 15:07:27.366361 2308 log.cc:824] T 00000000000000000000000000000000 P 39171c3b03544bf5b291757bc48b7021: Log is configured to *not* fsync() on all Append() calls
I20260806 15:07:27.367137 2308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39171c3b03544bf5b291757bc48b7021: No bootstrap required, opened a new log
I20260806 15:07:27.369089 2308 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39171c3b03544bf5b291757bc48b7021 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a735dfc7faf44634a35995bea09a8201" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46183 } } peers { permanent_uuid: "39171c3b03544bf5b291757bc48b7021" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 40801 } } peers { permanent_uuid: "96874dfb5bd4435bad58750969ef9c8e" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 } }
I20260806 15:07:27.369225 2308 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39171c3b03544bf5b291757bc48b7021 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39171c3b03544bf5b291757bc48b7021, State: Initialized, Role: FOLLOWER
I20260806 15:07:27.369297 2308 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39171c3b03544bf5b291757bc48b7021 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a735dfc7faf44634a35995bea09a8201" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46183 } } peers { permanent_uuid: "39171c3b03544bf5b291757bc48b7021" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 40801 } } peers { permanent_uuid: "96874dfb5bd4435bad58750969ef9c8e" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 } }
I20260806 15:07:27.369596 2308 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39171c3b03544bf5b291757bc48b7021 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:07:27.369982 2267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39171c3b03544bf5b291757bc48b7021 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:07:27.370826 2267 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 39171c3b03544bf5b291757bc48b7021 [term 1 FOLLOWER]: Refusing update from remote peer a735dfc7faf44634a35995bea09a8201: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:07:27.371061 2268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a735dfc7faf44634a35995bea09a8201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "39171c3b03544bf5b291757bc48b7021"
I20260806 15:07:27.371169 2455 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39171c3b03544bf5b291757bc48b7021" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 40801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:07:27.371788 2268 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 39171c3b03544bf5b291757bc48b7021 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a735dfc7faf44634a35995bea09a8201 in term 1.
I20260806 15:07:27.373934 2494 mvcc.cc:205] Tried to move back new op lower bound from 7315574158781669376 to 7315574158653341696. Current Snapshot: MvccSnapshot[applied={T|T < 7315574158667931648}]
I20260806 15:07:27.373620 2493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39171c3b03544bf5b291757bc48b7021 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a735dfc7faf44634a35995bea09a8201. Latest consensus state: current_term: 1 leader_uuid: "a735dfc7faf44634a35995bea09a8201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a735dfc7faf44634a35995bea09a8201" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46183 } } peers { permanent_uuid: "39171c3b03544bf5b291757bc48b7021" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 40801 } } peers { permanent_uuid: "96874dfb5bd4435bad58750969ef9c8e" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 } } }
I20260806 15:07:27.374390 2493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39171c3b03544bf5b291757bc48b7021 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:07:27.374459 2493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39171c3b03544bf5b291757bc48b7021 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a735dfc7faf44634a35995bea09a8201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a735dfc7faf44634a35995bea09a8201" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46183 } } peers { permanent_uuid: "39171c3b03544bf5b291757bc48b7021" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 40801 } } peers { permanent_uuid: "96874dfb5bd4435bad58750969ef9c8e" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 } } }
I20260806 15:07:27.374509 2493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39171c3b03544bf5b291757bc48b7021 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 15:07:27.376118 2500 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 39171c3b03544bf5b291757bc48b7021: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:07:27.376174 2500 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:07:27.376248 2475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39171c3b03544bf5b291757bc48b7021 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a735dfc7faf44634a35995bea09a8201" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46183 } } peers { permanent_uuid: "39171c3b03544bf5b291757bc48b7021" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 40801 } } peers { permanent_uuid: "96874dfb5bd4435bad58750969ef9c8e" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 } } }
I20260806 15:07:27.376318 2475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39171c3b03544bf5b291757bc48b7021 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 15:07:27.397116 2473 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:07:27.395415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59242 (local address 127.0.236.124:33191)
0806 15:07:27.395459 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.395461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.396462 (+ 1001us) server_negotiation.cc:407] Connection header received
0806 15:07:27.396566 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.396567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.396578 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.396607 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.397048 (+ 441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59242: BlockingRecv error: recv got EOF from 127.0.0.1:59242 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 15:07:27.397872 2207 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:07:27.395814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48256 (local address 127.0.236.126:46183)
0806 15:07:27.395855 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.395857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.397367 (+ 1510us) server_negotiation.cc:407] Connection header received
0806 15:07:27.397462 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.397463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.397474 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.397501 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.397812 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48256: BlockingRecv error: recv got EOF from 127.0.0.1:48256 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 15:07:27.398586 2316 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:07:27.396128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40330 (local address 127.0.236.125:40801)
0806 15:07:27.396164 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.396179 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.398101 (+ 1922us) server_negotiation.cc:407] Connection header received
0806 15:07:27.398192 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.398193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.398203 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.398230 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.398527 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40330: BlockingRecv error: recv got EOF from 127.0.0.1:40330 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 15:07:27.413427 2316 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:07:27.403151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40340 (local address 127.0.236.125:40801)
0806 15:07:27.403193 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.403195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.403374 (+ 179us) server_negotiation.cc:407] Connection header received
0806 15:07:27.412975 (+ 9601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.412977 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.412998 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.413025 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.413376 (+ 351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40340: BlockingRecv error: recv got EOF from 127.0.0.1:40340 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 15:07:27.419013 2316 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:07:27.418230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40348 (local address 127.0.236.125:40801)
0806 15:07:27.418265 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.418267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.418508 (+ 241us) server_negotiation.cc:407] Connection header received
0806 15:07:27.418595 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.418596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.418606 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.418632 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.418956 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40348: BlockingRecv error: recv got EOF from 127.0.0.1:40348 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 15:07:27.422843 2316 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:07:27.422143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40358 (local address 127.0.236.125:40801)
0806 15:07:27.422184 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.422186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.422396 (+ 210us) server_negotiation.cc:407] Connection header received
0806 15:07:27.422484 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.422485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.422494 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.422520 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.422802 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40358: BlockingRecv error: recv got EOF from 127.0.0.1:40358 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260806 15:07:27.428630 2316 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:07:27.427920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40370 (local address 127.0.236.125:40801)
0806 15:07:27.427952 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.427954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.428156 (+ 202us) server_negotiation.cc:407] Connection header received
0806 15:07:27.428242 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.428243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.428252 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.428278 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.428588 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40370: BlockingRecv error: recv got EOF from 127.0.0.1:40370 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 15:07:27.438560 2316 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:07:27.437783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40382 (local address 127.0.236.125:40801)
0806 15:07:27.437818 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.437820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.438063 (+ 243us) server_negotiation.cc:407] Connection header received
0806 15:07:27.438154 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.438155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.438165 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.438192 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.438514 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40382: BlockingRecv error: recv got EOF from 127.0.0.1:40382 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 15:07:27.456722 2316 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:07:27.455860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40392 (local address 127.0.236.125:40801)
0806 15:07:27.455901 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.455903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.456145 (+ 242us) server_negotiation.cc:407] Connection header received
0806 15:07:27.456241 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.456242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.456253 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.456282 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.456669 (+ 387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40392: BlockingRecv error: recv got EOF from 127.0.0.1:40392 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 15:07:27.490907 2316 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:07:27.490094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40406 (local address 127.0.236.125:40801)
0806 15:07:27.490139 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.490141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.490315 (+ 174us) server_negotiation.cc:407] Connection header received
0806 15:07:27.490399 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.490400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.490414 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.490479 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.490846 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40406: BlockingRecv error: recv got EOF from 127.0.0.1:40406 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 15:07:27.556820 2316 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:07:27.556150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40414 (local address 127.0.236.125:40801)
0806 15:07:27.556233 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.556236 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.556338 (+ 102us) server_negotiation.cc:407] Connection header received
0806 15:07:27.556372 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.556373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.556384 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.556441 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.556756 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40414: BlockingRecv error: recv got EOF from 127.0.0.1:40414 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 15:07:27.686590 2316 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:07:27.685939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40418 (local address 127.0.236.125:40801)
0806 15:07:27.686021 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.686024 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.686098 (+ 74us) server_negotiation.cc:407] Connection header received
0806 15:07:27.686152 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.686153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.686163 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.686213 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.686526 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40418: BlockingRecv error: recv got EOF from 127.0.0.1:40418 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 15:07:27.790146 2455 sys_catalog.cc:455] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a735dfc7faf44634a35995bea09a8201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a735dfc7faf44634a35995bea09a8201" member_type: VOTER last_known_addr { host: "127.0.236.126" port: 46183 } } peers { permanent_uuid: "39171c3b03544bf5b291757bc48b7021" member_type: VOTER last_known_addr { host: "127.0.236.125" port: 40801 } } peers { permanent_uuid: "96874dfb5bd4435bad58750969ef9c8e" member_type: VOTER last_known_addr { host: "127.0.236.124" port: 33191 } } }
I20260806 15:07:27.790304 2455 sys_catalog.cc:458] T 00000000000000000000000000000000 P a735dfc7faf44634a35995bea09a8201 [sys.catalog]: This master's current role is: LEADER
W20260806 15:07:27.944319 2316 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:07:27.943725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40424 (local address 127.0.236.125:40801)
0806 15:07:27.943797 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:27.943799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:27.943810 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:07:27.943861 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:27.943862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:27.943872 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:27.943949 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:07:27.944253 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40424: BlockingRecv error: recv got EOF from 127.0.0.1:40424 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 15:07:28.297452 2434 catalog_manager.cc:1259] Loaded cluster ID: 257147ba73bc41dfbd6c100709cfc3ae
I20260806 15:07:28.297520 2434 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 96874dfb5bd4435bad58750969ef9c8e: loading cluster ID for follower catalog manager: success
I20260806 15:07:28.298367 2434 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 96874dfb5bd4435bad58750969ef9c8e: acquiring CA information for follower catalog manager: success
I20260806 15:07:28.298667 2434 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 96874dfb5bd4435bad58750969ef9c8e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 15:07:28.376642 2500 catalog_manager.cc:1259] Loaded cluster ID: 257147ba73bc41dfbd6c100709cfc3ae
I20260806 15:07:28.376720 2500 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 39171c3b03544bf5b291757bc48b7021: loading cluster ID for follower catalog manager: success
I20260806 15:07:28.377524 2500 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 39171c3b03544bf5b291757bc48b7021: acquiring CA information for follower catalog manager: success
I20260806 15:07:28.377817 2500 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 39171c3b03544bf5b291757bc48b7021: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 15:07:28.459748 2602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:28.457418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40438 (local address 127.0.236.125:40801)
0806 15:07:28.457552 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:28.457554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:28.457565 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:07:28.457632 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:28.457633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:28.457687 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:28.457756 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:07:28.457876 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:28.458418 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:28.459118 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:28.459306 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:28.459357 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:28.459465 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:07:28.459575 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:07:28.459628 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:28.459647 (+ 19us) server_negotiation.cc:299] Negotiation successful
0806 15:07:28.459663 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":69,"threads_started":1}
I20260806 15:07:28.463809 2609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:28.461464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59258 (local address 127.0.236.124:33191)
0806 15:07:28.461589 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:28.461591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:28.461598 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:07:28.461637 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:28.461637 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:28.461681 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:28.461744 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:07:28.461851 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:28.462357 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:28.463063 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:28.463270 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:28.463323 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:28.463481 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:07:28.463585 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:07:28.463647 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:28.463703 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 15:07:28.463725 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":53,"threads_started":1}
I20260806 15:07:28.468309 2602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:28.465560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40454 (local address 127.0.236.125:40801)
0806 15:07:28.465598 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:28.465600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:28.465727 (+ 127us) server_negotiation.cc:407] Connection header received
0806 15:07:28.465849 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:28.465850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:28.465862 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:28.465931 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:07:28.466084 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:28.466796 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:28.467530 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:28.467813 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:28.467870 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:28.468047 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:07:28.468080 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:07:28.468125 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:28.468214 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 15:07:28.468237 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:07:28.468317 2619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:28.465421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48272 (local address 127.0.236.126:46183)
0806 15:07:28.465576 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:28.465578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:28.465586 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:07:28.465677 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:28.465678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:28.465736 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:28.465806 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:07:28.465989 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:28.466517 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:28.467496 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:28.467750 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:28.467811 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:28.467983 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:07:28.468088 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:07:28.468145 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:28.468207 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 15:07:28.468232 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":63,"threads_started":1}
I20260806 15:07:28.468556 2609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:07:28.465488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59270 (local address 127.0.236.124:33191)
0806 15:07:28.465532 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 15:07:28.465534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:07:28.465731 (+ 197us) server_negotiation.cc:407] Connection header received
0806 15:07:28.465775 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:07:28.465775 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:07:28.465786 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:07:28.465828 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:07:28.466048 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:28.466761 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:28.467722 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:07:28.467964 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:07:28.468024 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:07:28.468301 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:07:28.468331 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:07:28.468358 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0806 15:07:28.468468 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 15:07:28.468491 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 15:07:28.470228 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 2095
I20260806 15:07:28.482172 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 2213
I20260806 15:07:28.498951 945 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFIWuwX/build/release/bin/kudu with pid 2320
2026-08-06T15:07:28Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2878 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5749 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (267710 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST